Exchange 2016 CU20 breaks Event logs

I don't post often, but when I have something that I need to get out there with regards to my subject matter, this is where I post it. In this instance it relates to Exchange 2016 CU20 and a little known bug.

I found this quite by accident, many months after successful installation of CU20. I encountered no issues at all as it went on, but when I needed to find something in my event logs, it transpired that they had stopped being populated from the point of CU20 installation.

The log I am referring to is known as a crimson log. When you open Event Viewer and go to Applications and Services Logs and expand that section - these are crimson logs. The one I refer to in this post is the MSExchange Management log.

This log was broken and not just on one server. When an analysis was done, it had broken on 10% of my servers (4 out of 40) - which meant that when MS Premier Support tried to replicate the issue, they were able to do so quite easily.

To cut to the chase: the log is not actually broken. Rather, entries are being diverted into the Application Log instead (where they are easily lost due to the noise). The reason for this is that during CU20, setup may incorrectly write a registry key which takes precedence as the registry is read from top to bottom and the new key appears before the crimson log entries. The key is:

HKLM:\SYSTEM\CurrentControlSet\Services\EventLog\Application\MSExchange CmdletLogs\

It should not be there. Because it is, entries from source MSExchange CmdletLogs are routed to the Application Log instead of the MSExchange Management crimson log, which basically goes dormant. Microsoft don't have this listed on their database of eff-ups, but that may be because (like me) nobody looks at this log until they actually need to.

The fix is to remove the invalid key and restart the server.

To identify the presence of the key on your Exchange servers, you can use the following EMS:

Get-ExchangeServer | %{$s = New-PSSession -ComputerName $_.name; Invoke-Command -Session $s{if(Get-ItemProperty 'HKLM:\SYSTEM\CurrentControlSet\Services\EventLog\Application\MSExchange CmdletLogs\' -ErrorAction SilentlyContinue){$env:computername}}; remove-pssession $s}

If it finds it, you'll get the server NetBIOS name listed.

The following PowerShell will remove the key on a local server:

Remove-item 'HKLM:\SYSTEM\CurrentControlSet\Services\EventLog\Application\MSExchange CmdletLogs\' -Force -Recurse

That could be compiled into a remote PowerShell command, but I like to mitigate risk of trashing everything by taking such actions on a local server, which needs to be paced into Exchange Management Mode and restarted anyway (the key being in CurrentControlSet means that it only takes effect when the Registry hive is read i.e. on startup).

After a restart, the original command should create an error which is logged to the Event log you just fixed:

Get-ItemProperty 'HKLM:\SYSTEM\CurrentControlSet\Services\EventLog\Application\MSExchange CmdletLogs\' 

I don't know if this affects any other CUs on other Exchange versions as I am only currently working with Exchange 2016. I have implemented two further CUs since then and not experienced the same issue.

Y2K22

Tasked with recovering from the issue discovered in https://techcommunity.microsoft.com/t5/exchange-team-blog/email-stuck-in-exchange-on-premises-transport-queues/ba-p/3049447 (Email Stuck in Exchange On-premises Transport Queues) on a per server basis, I came up with the following PowerShell.

It's all well and good to provide an automated method, but most of us can't blindly run such scripts and hope that everything goes okay. It didn't go okay in the first place, hence why the issue needs fixing.

Some background:

A break/fix was put in place in conjunction with MS Premier Support:

Get-MalwareFilteringServer | Set-MalwareFilteringServer -BypassFiltering $true

Get-ExchangeServer | where { ($_.IsHubTransportServer -eq "true")} | ForEach{ Invoke-Command -ComputerName $_.Name -ScriptBlock { Restart-Service msexchangetransport } }

I wasn't involved in that (I was busy seeing in the New Year), but I get the point. The Malware Filtering was bypassed as it is only one of many layers of protection in this particular environment and the least important one. It's possibly the same elsewhere for other on-premises Exchange customers.

My mission (and I chose to accept it, as that is what I get paid for), was to fix the underlying issue before re-enabling the Malware Filtering. I don't normally have a need to do anything with it, but quickly observed that it is a bit 'laggy' when commands are run with it. So I wrote the following, which uses the manual rectification method in a set of commands that I can easily interact with and adjust if necessary. They are working well for me now, so I am finding that I can run them all in one hit and just observe. The updating takes at least half an hour per server and the use of timestamps is because I use a start-transcript for all my PS command windows.

Run from an elevated PowerShell prompt.

get-date -f HH:mm

$serverfqdn = ([System.Net.Dns]::GetHostByName($env:computerName)).HostName

. $env:ExchangeInstallPath\bin\RemoteExchange.ps1

Connect-ExchangeServer -auto -AllowClobber

$currentPrincipal = New-Object Security.Principal.WindowsPrincipal([Security.Principal.WindowsIdentity]::GetCurrent())

if($currentPrincipal.IsInRole([Security.Principal.WindowsBuiltInRole]::Administrator) -eq $FALSE){write-host -backgroundcolor yellow -foregroundcolor red "PLEASE RE-RUN AS ADMINISTRATOR"; EXIT}

$p = Get-Process -Name updateservice -ErrorAction SilentlyContinue

if($p){Stop-Process $p}

$path1 = $exinstall + "FIP-FS\Data\Engines\amd64\Microsoft"

$path2 = $exinstall + "FIP-FS\Data\Engines\metadata"

Remove-Item $path1 -Recurse -Force

Remove-Item $path2 -Recurse -Force

New-Item $path2 -Type Directory | out-null

get-service MSExchangeTransport | stop-service

get-Service FMS | start-service

get-service MSExchangeTransport | start-service

Add-PSSnapin Microsoft.Forefront.Filtering.Management.Powershell

cd $exscripts

$origcheck = (Get-EngineUpdateInformation).LastChecked

Get-EngineUpdateInformation

.\Update-MalwareFilteringServer.ps1 $serverfqdn

get-date -f HH:mm

while($origcheck -eq (Get-EngineUpdateInformation).LastChecked){sleep 15}

(Get-EngineUpdateInformation).UpdateStatus

get-date -f HH:mm

while((Get-EngineUpdateInformation).UpdateStatus -eq "UpdateInProgress"){sleep 60}; Get-EngineUpdateInformation

get-date -f HH:mm

There are a few things to note in this script.

1. A standard check that it is being run at an elevated prompt or stop you in your tracks

2. $exinstall allows for non-standard installation path. It is quite common

3. The while statements allow for the Malware update process to kick in. If it doesn't, you may have a proxy issue. Refer to the Exchange blog article to help you out on that one

4. Updating isn't quick. It takes between 30 and 60 minutes for me. But so far (touch wood) it has been successful every time.

5. This is just fixing the stuff under the cover. I will still need to re-enable the Malware Filtering service by setting the bypassfiltering to $false later

AMSI - Antimalware Scan Interface

Microsoft Exchange Server 2016 now supports integration with Windows Antimalware Scan Interface (AMSI). This feature enables an AMSI-capable antivirus or antimalware solution to scan content in HTTP requests that're sent to the Exchange Server. Additionally, it will block a malicious request before it's handled by Exchange. This was introduced with CU21 (also with Exchange Server 2019 CU10, or higher).

More information about AMSI is available at:

https://techcommunity.microsoft.com/t5/exchange-team-blog/more-about-amsi-integration-with-exchange-server/ba-p/2572371

It has been reported that in some environments users experience significant degradation in Outlook performance. I have only seen reports relating to third party Anti Virus products.

Whilst the feature is desirable to have, it may be that it has to be disabled until the performance issues are overcome.

The feature writes into AD at the following location:

CN=Setting Overrides, CN=Global Settings, CN=[your Exchange Org.], CN=Microsoft Exchange, CN=Services, CN=Configuration, DC=[your AD domain], DC=[your AD domain suffix]

So in ADSIEDIT:

Configuration > Services > Microsoft Exchange > [your Exchange Org.] > Global Settings > Setting Overrides

The override you can create can be a global or local (server) override. By not using the Server attribute (and therefore not defining it), you will be creating a Global override that applies to all applicable Exchange servers.

To create a Server Override for Server1 (needs to be performed on an upgraded server, e.g. Server1):

New-SettingOverride -Name "DisablingAMSIScan" -Component Cafe -Section HttpRequestFiltering -Parameters ("Enabled=False") -Reason "Testing" -Server Server1

Next, on Server1:

Get-ExchangeDiagnosticInfo -Process Microsoft.Exchange.Directory.TopologyService -Component VariantConfiguration -Argument Refresh

Finally (also on Server 1):

Restart-Service -Name W3SVC, WAS -Force

If you didn't set it as a local override in the first instance, you need to use the Add command and you need to specify the servers it applies to.

Add-SettingOverride DisablingAMSIScan -Server Server1, Server2

Once you have populated the servers attribute, you need to use the Set command to modify it. (I tried appending the attribute, unsuccessfully - as the property type is affected). So adding Server3:

Set-SettingOverride DisablingAMSIScan -Server Server1, Server2, Server3

Obviously, the post-config part could be done with remote PS:

$servers = @("Server1", "Server2", "Server3")

Foreach($server in $servers){$s = new-pssession -computername $server; invoke-command -session $s{Get-ExchangeDiagnosticInfo -Process Microsoft.Exchange.Directory.TopologyService -Component VariantConfiguration -Argument Refresh; Restart-Service -Name W3SVC, WAS -Force};Exit-PSSession}

Removing servers from the override is removing them from the defined servers and then running the recycling commands again. So to remove Server3:

Set-SettingOverride DisablingAMSIScan -Server Server1, Server2

Once again, the post-config part could be done with remote PS:

$servers = @("Server3")

Foreach($server in $servers){$s = new-pssession -computername $server; invoke-command -session $s{Get-ExchangeDiagnosticInfo -Process Microsoft.Exchange.Directory.TopologyService -Component VariantConfiguration -Argument Refresh; Restart-Service -Name W3SVC, WAS -Force};Exit-PSSession}

To remove the override completely, use the Remove-SettingOverride command.

Exchange Transport Rules and Regular expressions

 I haven't posted anything in a long time and I have chosen not to go back over what has already been posted to see if it is still valid. I expect readers to have a level of understanding and knowledge to know how to interpret what has been posted and determine if it is relevant. I use blogs like these just to point me in the right direction, which brings me to an item that Microsoft had published for Exchange 2010, but the link is now defunct (after they moved their library) and I couldn't find it anywhere else, except on another blog. So this is to bolster that blog in case it too vanishes.

When using PowerShell / Exchange Management Shell, certain characters are interpreted as scripting language (e.g. the $ symbol). When writing something to the screen for example, when a dollar sign is encountered, PS is looking for a variable. You can overcome this using the tilde character (`) immediately before any character you want to be displayed literally (it is located to the left of number one across the top of most keyboards). So, to get the following to display the way you intend it to:

write-host "The contents of $variable are"$variable

you would write:

write-host "The contents of `$variable are"$variable

It's a subtle difference. If you wanted to display a tilde, you'd put a tilde in front of it.

The same principle is not used in Transport Rules. When you want to match criteria in a rule, you need to use a different approach. Below is a copy of the content originally posted by Microsoft. It was in relation to Exchange 2010, but is still relevant in exchange 2016 at least. Note that the criteria you filter for is not case-sensitive so looking for 'External' is the same as 'external' and 'EXTERNAL' and even 'ExTeRnAL'.

Here is an example to match only 'External' encased in brackets and not 'external' elsewhere in the field being searched:

\(External\)

Pattern matching in Exchange Transport Rules

Pattern string	Description
\S	The \S pattern string matches any single character that's not a space.
\s	The \s pattern string matches any single white-space character.
\D	The \D pattern string matches any non-numeric digit.
\d	The \d pattern string matches any single numeric digit.
\w	The \w pattern string matches any single Unicode character categorized as a letter or decimal digit.
\W	The \W pattern string matches any single Unicode character not categorized as a letter or a decimal digit.
|	The pipe ( | ) character performs an OR function.
*	The asterisk ( * ) character matches zero or more instances of the previous character. For example, ab*c matches the following strings: ac, abc, abbbbc.
( )	Parentheses act as grouping delimiters. For example, a(bc)* matches the following strings: a, abc, abcbc, abcbcbc, and so on.
\	A backslash is used as an escaping character before a special character. Special characters are characters used in pattern strings: Backslash ( \ ) Pipe ( | ) Asterisk ( * ) Opening parenthesis ( ( ) Closing parenthesis ( ) ) Caret ( ^ ) Dollar sign ( $ ) For example, if you want to match a string that contains (525), you would type \(525\).
^	The caret ( ^ ) character indicates that the pattern string that follows the caret must exist at the start of the text string being matched. For example, ^fred@contoso matches fred@contoso.com and fred@contoso.co.uk but not alfred@contoso.com.
$	The dollar sign ( $ ) character indicates that the preceding pattern string must exist at the end of the text string being matched. For example, contoso.com$ matches adam@contoso.com and kim@research.contoso.com, but doesn't match kim@contoso.com.au

EWS & PowerShell

A bit of stuff using the EWS managed API, which can be downloaded here:
http://www.microsoft.com/en-us/download/details.aspx?id=35371

Note: Make sure the path to the EWS dll matches the install location of the EWS managed API. Change it if necessary.

This script searches folders in mailboxes under the Contacts folder, looking for 'Skype for Business contacts' and then deletes it. It is also doing a disable-ummailbox. As both actions are quite destructive, there's a warning for the person running the script.

The list of mailboxes to run this against was gathered with a get-aduser searchbase of specific OUs, extracting the Primary SMTP address and outputting that to a text file.


$inputfile = "mailboxlist.txt"

$MailboxList = get-content $inputfile
$MailboxListcount = $MailboxList.count

$CurrentDate = (Get-Date).ToString('MMM-dd-yyyy')

[string]$logfilepath = "SkypeForBus-DeletionsLog_" + $CurrentDate + ".csv"

Write-host -backgroundcolor yellow -foregroundcolor red "WARNING! This script will perform the following:"`n
write-host "Using the file $inputfile containing $MailboxListcount users"`n
write-host "Action 1: Disable-UMMailbox -Identity `$MailboxName -KeepProperties:`$True -Confirm:`$false"
write-host "Action 2: Delete the Skype for Business contacts folder (if found) under the Contacts folder"`n

$verify = Read-Host "If you wish to proceed, type Y"

if($verify -eq "Y"){

$dllpath = "C:\Program Files\Microsoft\Exchange\Web Services\2.0\Microsoft.Exchange.WebServices.dll"
    [void][Reflection.Assembly]::LoadFile($dllpath)
    $Service = New-Object Microsoft.Exchange.WebServices.Data.ExchangeService([Microsoft.Exchange.WebServices.Data.ExchangeVersion]::Exchange2010_SP1)

$FoundAmount = 0
$log = @()
$T1 = "Processing "
$T2 = " ("
$T3 = " of "
$T4 = ")"
$counter = 0
$MailboxListCount = $MailboxList.count

$StartTime = Get-Date

Foreach ($MailboxName in $MailboxList)
{
$counter++

$Activity = $T1+$MailboxName+$T2+$counter+$T3+$MailboxListCount+$T4

$percent = (($counter/$MailboxListCount) * 100)

$percentstatus = [String]([System.Math]::Round($percent, 0)) + "`% complete"

Write-Progress -id 1 -PercentComplete $percent -Activity $activity -Status $percentstatus

$FoundFolder = $FALSE
$i = New-Object -TypeName PSObject
$i | Add-Member NoteProperty Email $MailboxName
if(Get-UMMailbox $MailboxName){$UMdisabled = "True"
    Disable-UMMailbox -Identity $MailboxName -KeepProperties:$True -Confirm:$false
$i | Add-Member NoteProperty UMdisabled "True"
}
else{
$i | Add-Member NoteProperty UMdisabled "NA"
}

#    $Service.AutodiscoverUrl($MailboxName,{$true})
    $service.Url= new-object Uri("https://outlook.uk2.practice.linklaters.net/EWS/Exchange.asmx")
    $RootFolderID = new-object Microsoft.Exchange.WebServices.Data.FolderId([Microsoft.Exchange.WebServices.Data.WellKnownFolderName]::Contacts,$MailboxName)
    $RootFolder = [Microsoft.Exchange.WebServices.Data.Folder]::Bind($Service,$RootFolderID)
    $FolderView = New-Object Microsoft.Exchange.WebServices.Data.FolderView(100,0)
    $FolderView.Traversal = [Microsoft.Exchange.WebServices.Data.FolderTraversal]::Shallow
    $Response = $RootFolder.FindFolders($FolderView)
    

    ForEach ($Folder in $Response.Folders){
            if($folder.DisplayName -eq "Skype for Business contacts"){
        ++$FoundAmount
$FoundFolder = $True
#$folder.delete([Microsoft.Exchange.WebServices.Data.DeleteMode]::SoftDelete)
     #$folder.Empty([Microsoft.Exchange.WebServices.Data.DeleteMode]::SoftDelete, $true)
$i | Add-Member NoteProperty Folder $folder.DisplayName
$i | Add-Member NoteProperty Action "Deleted"
}
    }
    if($FoundFolder -eq $false){
$i | Add-Member NoteProperty Folder "NotFound"
$i | Add-Member NoteProperty Action "None"
        }
$log += $i
}

$log | export-csv $logfilepath -notypeinformation -append

$StopTime = Get-Date
$TimeDiff = New-TimeSpan -Start $StartTime -End $StopTime
Write-Host "Deleted"$FoundAmount "folders in"$TimeDiff " and saved log to: "$logfilepath
}

Exchange Database statistics

DB sizes with whitespace:

Get-MailboxDatabase -Status | sort name | select name,@{Name='DB Size (Gb)';Expression={$_.DatabaseSize.ToGb()}},@{Name='Available New Mbx Space Gb)';Expression={$_.AvailableNewMailboxSpace.ToGb()}} | ft -auto

Database usage stats:

$dbs = Get-mailboxdatabase | select Name,Server | sort Name

$collcount = $dbs.count

$T1 = "Processing "

$T2 = " ("

$T3 = " of "

$T4 = ")"

$counter = 0

$results=@()

foreach($db in $databases){

$counter++

$Activity = $T1+$T2+$counter+$T3+$collcount+$T4

$percent = (($counter/$userscount) * 100)

$percentstatus = [String]([System.Math]::Round($percent, 0)) + "`% complete"

Write-Progress -PercentComplete $percent -Activity $activity -Status $percentstatus

$SUS = Get-StoreUsageStatistics -Database $db.name

$i = New-Object -TypeName PSObject

$i | Add-Member NoteProperty Database $db.name

$i | Add-Member NoteProperty Server $db.server

$i | Add-Member NoteProperty DisplayName $SUS.DisplayName

$i | Add-Member NoteProperty DigestCategory $SUS.DigestCategory

$i | Add-Member NoteProperty TimeInServer $SUS.TimeInServer

$i | Add-Member NoteProperty TimeInServer $SUS.TimeInCPU

$i | Add-Member NoteProperty SampleId $SUS.SampleId

$results += $i

}

$results | sort Server, TimeInServer -descending, Database | export-csv database-statistics.csv -notypeinformation

Functions 1

############## function to add append to export-csv #########################

function Export-CSV {
[CmdletBinding(DefaultParameterSetName='Delimiter',
  SupportsShouldProcess=$true, ConfirmImpact='Medium')]
param(
 [Parameter(Mandatory=$true, ValueFromPipeline=$true,
           ValueFromPipelineByPropertyName=$true)]
 [System.Management.Automation.PSObject]
 ${InputObject},

 [Parameter(Mandatory=$true, Position=0)]
 [Alias('PSPath')]
 [System.String]
 ${Path},

 [Switch]
 ${Append},

 [Switch]
 ${Force},

 [Switch]
 ${NoClobber},

 [ValidateSet('Unicode','UTF7','UTF8','ASCII','UTF32',
                  'BigEndianUnicode','Default','OEM')]
 [System.String]
 ${Encoding},

 [Parameter(ParameterSetName='Delimiter', Position=1)]
 [ValidateNotNull()]
 [System.Char]
 ${Delimiter},

 [Parameter(ParameterSetName='UseCulture')]
 [Switch]
 ${UseCulture},

 [Alias('NTI')]
 [Switch]
 ${NoTypeInformation})

begin
{
 $AppendMode = $false

 try {
  $outBuffer = $null
  if ($PSBoundParameters.TryGetValue('OutBuffer', [ref]$outBuffer))
  {
      $PSBoundParameters['OutBuffer'] = 1
  }
  $wrappedCmd = $ExecutionContext.InvokeCommand.GetCommand('Export-Csv',
    [System.Management.Automation.CommandTypes]::Cmdlet)
        
 $scriptCmdPipeline = ''

if ($Append) {
  
  $PSBoundParameters.Remove('Append') | Out-Null
    
  if ($Path) {
   if (Test-Path $Path) {        
    # Need to construct new command line
    $AppendMode = $true
    
    if ($Encoding.Length -eq 0) {
     $Encoding = 'ASCII'
    }
    
    $scriptCmdPipeline += 'ConvertTo-Csv -NoTypeInformation '

    if ( $UseCulture ) {
     $scriptCmdPipeline += ' -UseCulture '
    }
    if ( $Delimiter ) {
     $scriptCmdPipeline += " -Delimiter '$Delimiter' "
    } 
    
    $scriptCmdPipeline += ' | Foreach-Object {$start=$true}'
    $scriptCmdPipeline += '{if ($start) {$start=$false} else {$_}} '
    
    $scriptCmdPipeline += " | Out-File -FilePath '$Path'"
    $scriptCmdPipeline += " -Encoding '$Encoding' -Append "
    
    if ($Force) {
     $scriptCmdPipeline += ' -Force'
    }

    if ($NoClobber) {
     $scriptCmdPipeline += ' -NoClobber'
    }   
   }
  }
 } 
  
$scriptCmd = {& $wrappedCmd @PSBoundParameters }

 if ( $AppendMode ) {
  $scriptCmd = $ExecutionContext.InvokeCommand.NewScriptBlock(
      $scriptCmdPipeline
    )
 } else {
  $scriptCmd = $ExecutionContext.InvokeCommand.NewScriptBlock(
      [string]$scriptCmd
    )
 }

$steppablePipeline = $scriptCmd.GetSteppablePipeline(
        $myInvocation.CommandOrigin)
 $steppablePipeline.Begin($PSCmdlet)

 } catch {
   throw
 }
    
}

process
{
  try {
      $steppablePipeline.Process($_)
  } catch {
      throw
  }
}

end
{
  try {
      $steppablePipeline.End()
  } catch {
      throw
  }
}
}

######################################################################

function folder-statistics ($mailbox){

if(!($mailbox)){write-host -backgroundcolor yellow -foregroundcolor red "You must specify mailbox"}

else{

$INC = read-host "Enter INC reference"

$CurrentDate = (Get-Date).ToString('_MMM-dd-yyyy_HH-mm-ss_')

$outfile = "c:\users\me\documents\Reports\folderstats_" + $mailbox + $CurrentDate + $INC + ".csv"

[INT]$reportvalue = Read-host "Specify folder size value to report above"

$newhash = @()

$mbsize = Get-MailboxStatistics $mailbox | select TotalItemSize

[String]$mbsizestring = $mbsize.TotalItemSize

$mbSizeMB = [math]::Round(((((($mbsizestring.split("(")[1])).replace(" bytes)","")).replace(",","")))/1048576)

$mbSizeGB = [math]::Round(($mbSizeMB/1024),2)

$folderstats = Get-MailboxFolderStatistics $mailbox | select identity,name,FolderPath,FolderType,ItemsInFolder,DeletedItemsInFolder,FolderSize,ItemsInFolderAndSubfolders,DeletedItemsInFolderAndSubfolders,FolderAndSubfolderSize

foreach($fs in $folderstats){

[string]$foldersize = $fs.foldersize

$FolderSizeMB = [math]::Round(((((($foldersize.split("(")[1])).replace(" bytes)","")).replace(",","")))/1048576)

$FolderSizeGB = [String][System.math]::Round(($FolderSizeMB/1024),2)

$percentofmailbox = (($FolderSizeMB/$mbSizeMB) * 100)

$percentrounded = [string]([System.Math]::Round($percentofmailbox,2))

[string]$identity=$fs.identity

$i = New-Object -TypeName PSObject

$i | Add-Member NoteProperty User $mailbox

$i | Add-Member NoteProperty TotalSize $mbSizeGB

$i | Add-Member NoteProperty Identity $identity.replace($mailbox,"")

$i | Add-Member NoteProperty Name $fs.name

$i | Add-Member NoteProperty GBFolderSize $FolderSizeGB

$i | Add-Member NoteProperty MBFolderSize $FolderSizeMB

$i | Add-Member NoteProperty PerCent $percentrounded

$i | Add-Member NoteProperty ItemsInFolder $fs.ItemsInFolder

$i | Add-Member NoteProperty DeletedItemsInFolder $fs.DeletedItemsInFolder

$i | Add-Member NoteProperty FolderPath $fs.FolderPath

if($FolderSizeMB -gt $reportvalue){

$newhash += $i

}

}

$newhash = $newhash | sort MBFolderSize -descending | export-csv $outfile -NoTypeInformation

start-process excel.exe $outfile

}

}

######################################################################

function enumerate-dl ($dl){

$users = Get-DistributionGroupMember $dl | ?{$_.RecipientType -like "Use*"} | select name | sort name

$groups = Get-DistributionGroupMember $dl | ?{$_.RecipientType -like "*Group"} | select name | sort name

if($groups){Write-Host -foregroundcolor yellow "Nested groups:"

$groups | ft -hidetableheaders}

else{Write-Host -foregroundcolor green "No nested groups found"}

if($users){Write-Host -foregroundcolor white "Users:"

$users | ft -hidetableheaders}

else{Write-Host -foregroundcolor yellow "No users found"}

}

######################################################################

function queues {

$Global:hubs = Get-TransportServer

foreach($hub in $hubs){get-queue -Server $hub | ?{$_.MessageCount -gt 1 -and $_.DeliveryType -notlike "shadow*"}}

}

######################################################################

One-liners

Re-vamping this blog a bit. You'll notice that I do often like to squeeze my scripts into one-liners a lot of the time.

Basic Progress bar

$T1="Processing ";$T2=" (";$T3=" of ";$T4=")";$cnt=0

$cnt++;$Act=$T1+$var.identity+$T2+$cnt+$T3+$vars.count+$T4;$pc=(($cnt/$vars.count)*100);$pcs=[String]([System.Math]::Round($pc,0))+"`% complete";Write-Progress -PercentComplete $pc -Activity $Act -Status $pcs

Here is an example. This will get the mailbox databases in the Org and check whether they are on their Activation Preference 1 and provide output to the screen. I have more elaborate scripts but this is just to show how the progress bar works. Exchange 2007 & later, EMS - PS 2.0

$T1="Processing ";$T2=" (";$T3=" of ";$T4=")";$cnt=0

$dbs=get-mailboxdatabase | ?{$_.ReplicationType -ne "None"} | sort -unique | sort name

foreach ($db in $dbs){

$cnt++;$Act=$T1+$db.name+$T2+$cnt+$T3+$dbs.count+$T4;$pc=(($cnt/$dbs.count)*100);$pcs=[String]([System.Math]::Round($pc,0))+"`% complete";Write-Progress -PercentComplete $pc -Activity $Act -Status $pcs

$mdb = Get-MailboxDatabase $db | Select ActivationPreference,Server
$DbPrefSvr=$mdb.ActivationPreference | ?{$_.value -eq 1}
$DbPref=$DbPrefSvr.key.name;$DbNowSrv=$mdb.Server.name

If ($DbPref -ne $DbNowSrv){write-host -foregroundcolor yellow $DB "is on" $DbNowSrv "but should be on" $DbPref}
}

View entire forest

$CheckADServerSettings = Get-AdServerSettings; if(($CheckADServerSettings).ViewEntireForest -ne $true){Set-ADServerSettings -viewentireforest $true}

This one is useful if you have a Forest Root and you're working from a sub domain. I get that more often than you'd think. Exchange 2007 & later, EMS - PS 2.0

Full mailbox access, send as

EMS:

FULL MAILBOX ACCESS

Add full mailbox access for a user:

$mailbox = Read-host "Mailbox to be actioned upon"; $user = Read-host "User to be assigned full mailbox access"; Add-MailboxPermission -User $user -AccessRights 'FullAccess' -Identity $mailbox

Add full mailbox access for a user (without automapping):

$mailbox = Read-host "Mailbox to be actioned upon"; $user = Read-host "User to be assigned full mailbox access"; Add-MailboxPermission -User $user -AccessRights 'FullAccess' -Identity $mailbox -Automapping $false

https://technet.microsoft.com/en-us/library/bb676551%28v=exchg.141%29.aspx

Remove full mailbox access for a user:

$mailbox = Read-host "Mailbox to be actioned upon"; $user = Read-host "User to have full mailbox access REMOVED"; Remove-MailboxPermission -User $user -AccessRights 'FullAccess' -Identity $mailbox -Confirm:$false

SEND AS

Add 'send as' permission for a user:

$mailbox = Read-host "Mailbox to be actioned upon"; $user = Read-host "User to be assigned send-as permission"; Get-Mailbox $mailbox | Add-ADPermission -User $user -Extendedrights "Send As" 

https://technet.microsoft.com/en-us/library/bb676368%28v=exchg.141%29.aspx

Remove 'send as' permission for a user:

$mailbox = Read-host "Mailbox to be actioned upon"; $user = Read-host "User to have send-as permission REMOVED"; Get-Mailbox $mailbox | Remove-ADPermission -User $user -Extendedrights "Send As" -Confirm:$false

Monitor message queues & alert

EMS:


[string]$smtpserver = <HTServer>
[string]$recipient = <e-mail of recipient for alerts>

[string]$hostserver = $env:computername
[string]$sender = $hostname + "@" + ($recipient -split ("@"))[1]

write-host -foregroundcolor Cyan "This script will monitor message queues and create an alert if the queue message count threshold is exceeded"
$threshold = read-host "Specify a value for the threshold"
write-host -foregroundcolor Cyan "The timeout between messages is configurable"
$timeout = read-host "Specify a timeout in seconds"
$mins = $timeout/60
$HTS = (Get-ExchangeServer | ?{$_.IsHubTransportServer -eq "True"})
while($true){foreach ($HT in $HTS){Get-queue -server $HT | ?{$_.MessageCount -gt $threshold -and $_.DeliveryType -ne "ShadowRedundancy"}
$Queue = Get-queue -server $HT | ?{$_.MessageCount -gt $threshold -and $_.DeliveryType -ne "ShadowRedundancy"}
if ($Queue.MessageCount -gt $threshold -and $Queue.DeliveryType -ne "ShadowRedundancy"){write-host -foregroundcolor yellow "Sent mail"; $MSGQueue = $Queue | out-string; Send-MailMessage -To $recipient -From $sender -Subject "Queues alert - threshold=$threshold, timeout to next alert $mins minute(s)" -body "A message queue has triggered an alert: $MSGQueue" -SmtpServer $smtpserver; sleep $timeout}}; write-host -foregroundcolor green "Script running... Timeout="$timeout" second(s), Queue message threshold= "$threshold; sleep 5}

Show Queues

EMS one liner:

$HTS = (Get-ExchangeServer | ?{$_.IsHubTransportServer -eq "True"}); while ($true){foreach ($HT in $HTS){Get-queue -server $HT | ?{$_.MessageCount -gt "0" -and $_.DeliveryType -ne "ShadowRedundancy"}}; write-host -foregroundcolor green "Next..."; sleep 15}

Database Size Limit in GB script

EMS to set the database size limit in the registry

$DBsize = "200"
$loc = get-location
[string]$server = $env:computername

$var1 = "HKLM:\System\CurrentControlSet\services\MSExchangeIS\"
$var2 = "\"
$var3 = "Database Size Limit In GB"
$var4 = “*Database Size Limit In GB*”
$var5 = "\Logstate"

$regkeys = (Get-ChildItem -path $var1$server)

Set-location -path REGISTRY::

foreach ($regkey in $regkeys){
$key = (Get-ItemProperty -path $regkey); if ($key -notlike $var4){New-ItemProperty -path $regkey -name $var3 -value $DBsize -PropertyTYpe "DWord"}
}

Remove-itemProperty $var1$server$Var5 -name $var3 -confirm:$false

set-location -path $loc 

http://technet.microsoft.com/en-us/library/bb232092(v=exchg.141).aspx

Method not allowed

Method not allowed

Whilst trying to perform DAG tasks in an Exchange 2010 environment (suspending replication, moving an active database copy), I was getting some warnings (in yellow) as follows:

WARNING: The cmdlet extension agent with the index 0 has thrown an exception in OnComplete(). The exception is:
System.Net.WebException: The request failed with HTTP status 405: Method Not Allowed.
at System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage message, WebResponse response, Stream responseStream, Boolean asyncCall)
at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)
at Microsoft.Exchange.SoapWebClient.CustomSoapHttpClientProtocol.<>c__DisplayClass4.b__3()
at Microsoft.Exchange.SoapWebClient.HttpAuthenticator.NetworkServiceHttpAuthenticator.AuthenticateAndExecute[T](SoapHttpClientProtocol client, AuthenticateAndExecuteHandler`1 handler)
at Microsoft.Exchange.SoapWebClient.SoapHttpClientAuthenticator.AuthenticateAndExecute[T](SoapHttpClientProtocolclient, AuthenticateAndExecuteHandler`1 handler)
at Microsoft.Exchange.SoapWebClient.EWS.ExchangeServiceBinding.FindFolder(FindFolderType FindFolder1)
at Microsoft.Exchange.ProvisioningAgent.MailboxLoggerFactory.EwsMailer.GetAdminAuditLogsFolder(ADUser adUser)
at Microsoft.Exchange.ProvisioningAgent.MailboxLoggerFactory.EwsMailer..ctor(OrganizationId organizationId, ADUser adUser, ExchangePrincipal principal)
at Microsoft.Exchange.ProvisioningAgent.MailboxLoggerFactory.Create(OrganizationId organizationId, ADUser mailbox, ExchangePrincipal principal)
at Microsoft.Exchange.ProvisioningAgent.AdminLogAgentClassFactory.ConfigWrapper.get_MailboxLogger()
at Microsoft.Exchange.ProvisioningAgent.AdminLogProvisioningHandler.OnComplete(Boolean succeeded, Exception e)
at Microsoft.Exchange.Provisioning.ProvisioningLayer.OnComplete(Task task, Boolean succeeded, Exception exception)

They were just warnings, but I still needed to get to the bottom of them as they indicate a problem.

The crux of this is the “The request failed with HTTP status 405: Method Not Allowed” – implying to me an authentication problem. I was suspecting that the hardware load balancers were at fault. I understood that the action that I was doing would have result in Autodiscover being updated (or attempted updating). I examined the authentication types on the Autodiscover virtual directory and tested direct connectivity by tricking my client with a hosts file entry. This ruled out the load balancers as I still got the error. Nothing was amiss with the vdirs or authentication. This was also not limited to one CAS server, thus reducing the likelihood of it being an IIS that simply needed recycling. I had deliberately set the external url on the Autodiscover virtual directory when I installed Exchange 2010 into the environment so that the existing clients didn’t get any impact from an untested service. That testing has been successfully completed, so it is now safe to expose clients to Autodiscover and that is what fixed it.

In my case the above warning message actually meant “The external url of the Autodiscover Virtual Directory is blank”. Obvious really, eh? I populated the external url for all the new Exchange 2010 CAS servers and that’s the end of the warnings.

Relay Connectors & multi-valued properties

I’ve been working with Receive connectors, specifically relay connectors.

101 Relay Connectors – A basic Receive Connector can be used to allow hosts to relay internally. By adding an AD Permission, you can allow hosts to relay externally. So why not create two relay connectors?

New-ReceiveConnector -Server -RemoteIPRanges 1.2.3.4 -Name "Allowed to Relay Internally HTS_Server" -AuthMechanism Tls, BasicAuth -PermissionGroups AnonymousUsers -MaxMessageSize 20MB -Bindings 0.0.0.0:25,:::25

New-ReceiveConnector -Server -RemoteIPRanges 6.7.8.9 -Name "Allowed to Relay Externally HTS_Server" -AuthMechanism Tls, BasicAuth -PermissionGroups AnonymousUsers -MaxMessageSize 20MB -Bindings 0.0.0.0:25,:::25

Get-ReceiveConnector | ?{$_.name -like "Allowed to Relay Externally*"} | Add-ADPermission -User "NT AUTHORITY\ANONYMOUS LOGON" -ExtendedRights "ms-Exch-SMTP-Accept-Any-Recipient"

So now we have Relay Connectors, but adding IP Addresses to them is not such a simple task. When you use Set-ReceiveConnector –RemoteIPRanges the existing values get overwritten.

I tried to apply the instruction given in article http://technet.microsoft.com/en-gb/library/bb684908(v=exchg.150).aspx as follows:

Set-ReceiveConnector <RelayConnector> –RemoteIPRanges @{Add=”1.2.3.4”}

It should have worked. But it didn’t :-(

So I tried this instead: http://exchangepedia.com/2007/02/how-to-update-multi-valued-attributes-in-powershell.html as follows:

$var= Get-ReceiveConnector <RelayConnector>; $var.RemoteIPRanges +=”1.2.3.4”; $var | Set-ReceiveConnector "RelayConnector"

And even:

$var= Get-ReceiveConnector <RelayConnector>; Get-Content .\IPs.txt | foreach {$rc.RemoteIPRanges += "$_"}; $var | Set-ReceiveConnector

Still no joy :-(

I don't want to have to do this in EMC. I may have a load of addresses to add. And the trouble with listing them is that the buffer size of the Exchange Management Shell causes the field to be truncated. Even after using Out-String. So what’s the quickest and easiest way to append IP addresses to a receive connector?
By using the ‘View Exchange Management Shell Command Log’ (http://technet.microsoft.com/en-gb/library/ee332355.aspx). If you add one IP address (e.g. 1.2.3.4) then you’ll see the command that was run to add it, listing all the existing values. Copy, paste, massage in notepad and use in EMS to add the additional IP addresses or ranges. E.g.:

Set-ReceiveConnector -Identity 'SERVER01\Allowed to Relay Internally SERVER01' -RemoteIPRanges '1.2.3.4', '2.3.4.5'

And then to list multiple IP addresses or address ranges in a Receive Connector's RemoteIPRanges property:

(Get-ReceiveConnector "RelayConnector").RemoteIPRanges | ft Lowerbound,Upperbound,RangeFormat -AutoSize