Attention Veeam and Microsoft Exchange 2016 users; if you are running Veeam B&R against your Microsoft Exchange 2016 environment, you may want to wait to upgrade to Exchange Cumulative Update 5.
As noted in Veeam Digest, customers who upgraded to CU5 are experiencing restore failures while utilizing Veeam Explorer for Exchange.
The specific restore failures pertain to mailbox item-level recoveries, which are failing due to a significant change in the mailbox database structure. Typically, users receive an error stating Invalid property BLOB.
Veeam is in the process of updating Veeam Explorer for Exchange to be able to parse the new blob format correctly. They plan to include the fix in Veeam Backup & Replication 9.5 Update 2.
Last Thursday, VMware distributed a rush post regarding an issue with Windows Server and VMware Tools. The issue pertains to packets being dropped on virtual machines running Windows as the guest OS and VMware Tools versions 9.10.0 through 10.1.5. For reference, 9.10.0 through 10.1.5 were packaged with ESXi 6.0 and 6.5, respectively. The increased rate of dropped packets stems from a malfunction in Windows Receive Side Scaling (RSS) on Windows 2012 and later guests. RSS normally works to distribute network processing across multiple CPUs. However, the fault in RSS causes all network traffic to run through one CPU on these Windows guests.
VMware notes that this situation results in performance degradation for the applications and services hosted on the affected virtual machine. I have already seen a post of this possibly affecting Veeam backup proxies and subsequent jobs. These proxies already experience extremely busy CPU’s and when compounded with this issue could be the cause intermittent job failures.
VMware is working on a fix and recommends downgrading VMware Tools to version 9.4.15 if you are experiencing the issue.
Attention VMware and Veeam backup users: If you are thinking about upgrading your VMware environment to 6.5, Veeam has a YouTube session for you. Veeam’s own Anton Gostev will be discussing ‘vSphere 6.5 backup gotchas’. As per the Veeam Community Digest, the session will be available to the base install group via his live YouTube channel tomorrow. A recording will be available for all others. In this session, he will be discussing topics that should be considered when upgrading to vSphere 6.5. Topics will include changes to encrypted VMs, NBD transports, VMFS 6, networkless guest processing along with additional information to ensure successful backups. This is a must watch for those weighing the idea of upgrading.
To receive Gostev’s weekly digest, sign up for access to Veeam Community Forums.
Update: Here is a link to the recorded session. Definitely worth checking out if you are contemplating an upgrade. Lots of important information to consider, specifically;
- Transport modes and performance degradation
- Network Backup mode is forced into NBDSSL resulting in degraded backup times.
- SESparse and Hot-Add requiring a considerable amount of IOPs also degrading performance.
- Inability to enable CBT on machines with snapshots
- vSphere Tags
- Encrypted VMs
- KMS required.
- Loss of transport modes.
Update – This Technet article has more information on installing KB3216755 as a critical bug fix for Windows Server 2016 Data Deduplication users
Server 2016 and Veeam users, an issue regarding NTFS volumes with deduplication enabled has recently arose. According to a Veeam Forum Digest, there has been reports that backup files residing on Server 2016 NTFS volumes with data deduplication enabled are becoming corrupt. The reports have indicated that large files (> 2.2TB) residing on these volumes are affected. It appears that Microsoft is aware of the issue and working on a patch. A few contributors have noted that there is a package in pre-release preview to address the issue (KB3216755). Current workarounds include moving to a ReFS volume or disable deduplication. Veeam has a thread on the issue.