PowerShell – New-ComplianceSearch script to go through all mailboxes, find a target message, and remove it

Here is another more advanced script that searches for messages of interest, then optionally exports and purges them.

So, what makes this script more advanced? Unlike the other script that uses Search-Mailbox ( https://365basics.com/powershell-search-mailbox-script-to-go-through-all-mailboxes-find-a-target-message-and-remove-it/ ) this script below is based on New-ComplianceSearch cmdlet:

Pros:

  • Super fast, multi-threaded, takes about 1-2 minutes on average to complete the whole task.
  • As Microsoft is trying to depreciate Search-Mailbox, they put more effort into New-ComplianceSearch, so we can expect new functionality to come in the future.
  • Can create an export file that can be downloaded and investigated.

Cons:

  • Always searches through all folders including Purged Items which might create some confusion when showing results (if running the same search again) – see UPDATE 9/30/2019
  • Cannot copy messages to another mailbox for you to investigate.

Of course there are quite a few scripts out there that use New-ComplianceSearch, everyone has their own approach. So let me show you how my script stands out and why it is worth for you to check it:

  • easily select and search for messages received today or between a specified date and today;
  • an option to provide details by yourself (sender’s email address, part of the subject line) or see all received messages for a specific user to faster select (just typing its number) a message of interest;
  • shows all progress and how much time each step took to complete;
  • displays search results before you take actions;
  • creates a single New-ComplianceSearch instance (rather than many for each user);
  • all actions at once – searches, exports, and purges;
  • deletes all old search, export, and purge actions older than 180 days (adjust to your need) but only those created by this very script.

UPDATE 9/30/2019:

I updated the script to exclude Purged Items from the search. However, there needs to be some preparation done (please see Auxiliary script to populate CustomAttribute10 with a Purged Items FolderID for more details).

###
$retentiondays = 180 # number of days to keep the results of the Compliance Search - applies to Search, Export, and Purge results. Adjust if needed.
###

$elapsed = [System.Diagnostics.Stopwatch]::StartNew()

Write-Host "When was that message received?"
Write-Host "1. Today."
Write-Host "2. I will specify the date."`n

$choice1 = Read-Host "Please select an option that fits your requirements the best ( 1 or 2 )"

while("1","2" -notcontains $choice1) {
	
	$choice1 = Read-Host "Please select an option that fits your requirements the best ( 1 or 2 )"

}

If ($choice1 -eq '1') {

	$startdate = (Get-Date).Date

}

ElseIf ($choice1 -eq '2') {

	$startdate = Read-Host `n"Please specify the date. The search will be done from that date"

}

Write-Host `n"Two options to choose from:"
Write-Host "1. I have all information I need for a manual input: the sender and the subject."
Write-Host "2. I know which user got the message of interest: all information will be used automatically once that message is located by me."`n

$choice2 = Read-Host "Please select an option that fits your requirements the best ( 1 or 2 )"

while("1","2" -notcontains $choice2) {
	
	$choice2 = Read-Host "Please select an option that fits your requirements the best ( 1 or 2 )"

}

If ($choice2 -eq '1') {

	$sender = Read-Host `n"Sender's email address"
	$subject = Read-Host "Part of the subject line"

}

ElseIf ($choice2 -eq '2') {
	
	$enddate = get-date
		
	$emailaddress = Read-Host `n"Email of the user that got the message of interest"

	$collection = @()
	
	$i = 0

	$emails = Get-MessageTrace -StartDate $startdate -EndDate $enddate -RecipientAddress $emailaddress | Select-Object Received,SenderAddress,Subject

	ForEach ($email in $emails) {

		$outObject = "" | Select Number,Received,"Sender Address",Subject
	
		$i = $i + 1    

		$outObject."Number" = $i
		$outObject."Received" = $email.Received.ToLocalTime()
		$outObject."Sender Address" = $email.SenderAddress
		$outObject."Subject" = $email."Subject"
	
		$collection += $outObject

	}

	$collection | Out-Host

	Do {
		Try {
			$num = $true
			[int]$selectedinput = Read-host "Select an email by typing its number"
		}
		Catch {$num = $false}
	}
	Until (($selectedinput -gt 0 -and $selectedinput -le $collection.count) -and $num -eq $true)

	$selectedemail = $collection[[int]$selectedinput - 1]

	$selectedemail

	$sender = $selectedemail.'Sender Address'
	$subject = $selectedemail.Subject
	
}

$today = get-date -Format yyyy/MM/dd_HH-mm-ss
$searchname = "Exchange_"+$today

$allmailboxescustomattribute10 = (get-mailbox -ResultSize unlimited | Where-Object {$_.CustomAttribute10 -ne ""}).CustomAttribute10

$tempquery = "Subject:""" + $subject + """" + " AND "+ "received>=""" + $startdate + """ AND " + "From=" + $sender
$query = $tempquery + " NOT (" + $($allmailboxescustomattribute10 -join ' OR ') + ")"

Write-Host "Your search query (excluding 'Purged Items' by default):"`n$tempquery`n

New-ComplianceSearch -Name $searchname -ExchangeLocation all -ContentMatchQuery $query | fl @{Name = "Compliance Search Name"; Expression = {$_.Name}}

Start-ComplianceSearch -Identity $searchname

Write-Host "Search request has been created"

$k = 0
Do {
	$k = $k + 1
	Write-Host "." -NoNewline -ForegroundColor Cyan
	Start-Sleep -Seconds 1
}
While ((Get-ComplianceSearch -Identity $searchname).Status -ne "Completed")

Write-Host `n"Search complete within "$k" seconds." -ForegroundColor Green

$compliancesearch = Get-ComplianceSearch -Identity $searchname

$foundresults = $compliancesearch.SuccessResults

$array = $foundresults.Split([Environment]::NewLine,[System.StringSplitOptions]::RemoveEmptyEntries) | Where {
	$_ -notlike "*Item count: 0*"
	}

Write-host `n"Search results:"

If ($array.Count -eq 0) {
	Write-Host "0 items have been found, try another queue."
}

Else {
	$array

	$answer = Read-Host `n"Would you like to export those messages from the list above and then purge them ( y / n )?"

	while("y","n" -notcontains $answer) {$answer = Read-Host "Would you like to export those messages from the list above and then purge them ( y / n )?"}

	If ($answer -eq 'y') {

		New-ComplianceSearchAction -SearchName $searchname -Export -ExchangeArchiveFormat SinglePst -Format FXStream | fl Name,SearchName,Action,RunBy


		Write-Host "Export request has been created"

		$k = 0
		Do {
			$k = $k + 1
			Write-Host "." -NoNewline -ForegroundColor Cyan
			Start-Sleep -Seconds 1
		}
		While ((Get-ComplianceSearchAction -Identity $searchname"_Export").Status -ne "Completed")

		Write-Host `n"Export complete within "$k" seconds." -ForegroundColor Green
		

		New-ComplianceSearchAction -SearchName $searchname -Purge -PurgeType HardDelete | fl Name,SearchName,Action,RunBy
	
		Write-Host "Purge request has been created"

		$k = 0
		Do {
			$k = $k + 1
			Write-Host "." -NoNewline -ForegroundColor Cyan
			Start-Sleep -Seconds 1
		}
		While ((Get-ComplianceSearchAction -Identity $searchname"_Purge").Status -ne "Completed")

		Write-Host `n"Purge complete within "$k" seconds." -ForegroundColor Green

		Write-Host `n"To download the exported data, please go to https://protection.office.com/contentsearchbeta?ContentOnly=1 / Export tab."
	   
	}

}

Write-Host `n"Clearing only Exchange_date_time Search, Export, and Purge compliance search results older than" $retentiondays "days."

Get-ComplianceSearch | Where-Object {
	$_.JobEndTime -lt (Get-Date).AddDays(-$retentiondays) -and
	$_.Status -eq "Completed" -and
	$_.Name -match '\d\d\d\d/\d\d/\d\d_\d\d-\d\d-\d\d'
} | Remove-ComplianceSearch -Confirm:$false

Get-ComplianceSearchAction | Where-Object {
	$_.JobEndTime -lt (Get-Date).AddDays(-$retentiondays) -and
	$_.Status -eq "Completed" -and
	(
		$_.Name -match 'Exchange_\d\d\d\d/\d\d/\d\d_\d\d-\d\d-\d\d_Purge' -or
		$_.Name -match 'Exchange_\d\d\d\d/\d\d/\d\d_\d\d-\d\d-\d\d_Export'
	)
} | Remove-ComplianceSearchAction -Confirm:$false

Write-Host `n"Complete"`n -ForegroundColor Green

Write-Host "Total Time: $($elapsed.Elapsed.ToString())"

Remove-Variable * -ErrorAction SilentlyContinue

This Post Has 16 Comments

  1. Branislav

    Your comment is awaiting moderation.

    Hello,
    I have two questions:
    Can I use some of this coding just to search for mails in Archive folder (Outlook 365)?
    Do you know what is best way (or any other way) to search Archive folder (Outlook 365)?
    Thank you,
    Branislav

    1. Pavel Bludov

      Hello Branislav,
      Thanks for checking out my blog.
      Do you mean an Archive folder that comes standard in each mailbox (even though it has nothing to do with an actual Archive) OR an Online Archive (the one you have to turn on for each user)?
      If just the folder, then yes.
      If the Online Archive, I will have to double check.

  2. Branislav

    It is Online Archive. As per my understanding, after 100 GB, in main inbox, the auxiliary mailbox (archive) is created. The problem is, that kind of folder is not included in regular search.
    Thank you very much for quick answer,
    Branislav

    1. Pavel Bludov

      Branislav,
      I don’t think anything will be created automatically. I just know that an Archive folder is nothing but a folder that is counted against your mailbox size quota.
      I did check though how the New-ComplianceSearch works and it does include a real Online Archive in its search – in the search/export results a main mailbox is marked with a “Primary” label, an Online Archive – a “MainArchive” label.
      Hope it helps.

  3. Dan

    Hi Pavel,

    THank you for this awesome script! I have one issue I wonder if anyone has a solution for. While I’m sitting at “Search request has been created” it takes 388 seconds to complete for 1 email sent today. And my session with exchange times out and I cannot export or purge because commands do not exist at that time. Anybody know how to extend this timeout?

    1. Pavel Bludov

      Hi Dan,
      You are welcome!
      Why does it take so long? I would say on average it takes about 10-20 seconds to go through about 200-300 mailboxes.
      Thanks!

      1. Dan

        Thanks for the reply. I am not sure why it takes so long. The targeted email was back in March. It was in 3 users mailboxes. There are appr 35k mailboxes and they all sit in O365. I managed to work up a pssessionoption to extend all timeouts long enough to let the script finish. But it still took Total Time: 00:25:25.7532004 Idk. I can setup 3 of these and let them run and still be better than doing each step manually so again, tyvm!

  4. Sam

    Hi Pavel,

    New-mailboxsearch would let you export search results to a target mailbox however I dont see this option for new-compliansearch so as to export search results to a target mailbox folder. Is there a way to accomplish this ?

    Please advise.
    Sam

    1. Pavel Bludov

      Hi Sam,
      I don’t think my script exports anything to a mailbox – it just creates an export entity which you are able to download if needed.

  5. Dunn

    Compliance search does not seems to work. When I choose to create search from specific date, it create a query with a lot of NOT ,,, and so on. When search is done, the portal shows the query with an error.

    1. Paul Bludov

      Hi Dunn,
      It should work fine but likely doesn’t because you decided to opt out from this another script – https://365basics.com/powershell-auxiliary-script-to-populate-customattribute10-with-a-purged-items-folderid/
      To fix that (and not use the 2nd script) you should adjust $query to $tempquery in this #107 line:
      New-ComplianceSearch -Name $searchname -ExchangeLocation all -ContentMatchQuery $query | fl @{Name = “Compliance Search Name”; Expression = {$_.Name}}

  6. Jon

    Hi Pavel,
    I have a scenario where some of the search results have 1000’s of emails which I need to remove and the limit of 10 emails is a problem. Do you know if it’s possible to loop a purge script that will remove all the emails when there are 100’s or 1000’s of results, so that all emails are deleted?

    Many thanks

  7. No One

    My ComplianceSearch is not returning any results. How can I check if I have the correct permissions? If the New-ComplianceSearch and Start-ComplianceSearch cmdlets work successfully, is it safe to say I have the correct permissions?

    1. Paul Bludov

      Hey,
      You can visit Microsoft Purview (former Compliance Admin Portal) and check the results of your search, cause that’s what this script does. You can also try and create your own compliance search requests using UI.

Leave a Reply