HPE 3PAR Adaptive Optimization configuration demystified

Adaptive Optimization is disk tiering technology within 3PAR OS, which automatically moves the most frequently accessed data blocks to the fastest disks and infrequently accessed data blocks of data to slower disks.

I will not try to recite official documentation to explain technology or best practices because I think official documentation from HPE does it very well. Where I see there is a huge confusion and lack of understanding from users is how to properly configure AO in production. Official documentation will leave very confused at best.

In this post I will show with screenshots how AO could be used in a real world example.

Continue reading

Posted in 3par | Tagged , , | 6 Comments

3PAR iSCSI Delayed ACK read or write performance issues

I’ve noticed a few people running iSCSI 3PARs with Delayed ACK enabled and experiencing latency on their datastores. Here’s what happens when you disable it (SSD backed datastore):
Before and after:

VMware KB about it:
https://kb.vmware.com/kb/1002598

I suggest to use Host Profiles to check compliance and make sure it’s disabled across all your hosts. In case you don’t use them I’ve also created Powercli script to disable it on all hosts.
Continue reading

Posted in 3par, Powershell, VMware | Tagged , , | 1 Comment

Brocade 300 password recovery and factory reset

Let’s say you purchase a couple of 300s on the Ebay and someone forgot to reset config but you can’t get in as all default passwords got changed. Here’s what we need to do:
Continue reading

Posted in Random stuff | Tagged , , , | 31 Comments

vCenter Server Appliance 6.0 bug – Intermediate authentication issues – Native platform error -1765328378

Another mysterious bug from VMware – intermediate authentication failures.

Symptoms: I’ve noticed some of the backup jobs were failing because Veeam failed to log in to vCenter. If you’re familiar with Veeam software – you define backup account per vCenter and not individual jobs. Hence if the account was invalid, didn’t have permissions it would affect every single job but not some. After troubleshooting I’ve discovered in vCenter logs was full of unsuccessful login attempts. Once I’ve contacted VMware support they confirmed to be a bug.

Continue reading

Posted in VMware | Tagged , , , | Leave a comment

3PAR with AO and SSDs. How much do I need?

Most of the people involved with storage deal on daily basis with performance and capacity monitoring. There are two metrics which everyone cares the most: size and IOPS. Of course, latency, queue depth and many more have significant impact as well. If you worked in enterprise storage you will notice that most of the time you will run out of IOPS before capacity.

HP 3PAR has several functionalities to help you get the most of your investment and today we will talk about Adaptive Optimization (AO).

This case will apply to hybrid array, such as the one having mixed of FC and SSD drives. In the example below I will help you understand how to calculate required capacity for each tier for a given application using 3PAR SSMC 3.0.

Continue reading

Posted in 3par | Tagged , , | Leave a comment

AWS Certified Solutions Architect – Associate Exam 50% off

Amazon has amazing promotion for AWS Certified Solutions Architect Associate exams – 50% off! So here’s the catch – you will get random questions which help Amazon in future AWS exams. These “test” questions are not graded and do not contribute towards your overall score. Additionally, Amazon will extand your exam by 30 minutes to accommodate extra questions.

Just log into webassessor portal and select exam with PR000128 code.

Good luck!

Posted in Random stuff | Tagged , , | 2 Comments

3PAR 3.2.2 MU3 and Veeam B&R 9.0 Update 2 – integration bug

Latest 3PAR firmware 3.2.2 MU3 breaks SAN integration with Veeam Backup and Replication. Issue affects all versions, including version 9 Update 2.

Symptoms
You job will fail with “User authentication failed” error:
bug
Continue reading

Posted in 3par, Random stuff | Tagged , , | 2 Comments

ESXi 5.5/6.x bug HPE CIM – /var/run/sfcb inode table of its ramdisk is full

Another bug from VMware/HPE – unfortunately we don’t have public KB available at this point. As per our conversation with VMware engineer this issue affects both ESXi 5.5 and ESXi 6.x hosts.
I suspect VMware sfcb service fails to clear temporary files or HPE CIM providers create files which they are not suppose to.
I observed this issue with HPE ProLiant BL660c Gen8 blades running ESXi 5.5. These blades come with 4 CPU sockets and 1TB of ram – they are hosting VDI environment so they do have high density and a lot of power on/off operations.
As the troubleshooting options we tried updating to the latest ESXi patches, HPE drivers and software but issue was still persisting.

Scope
Issue affects ESXi 5.5 and ESXi 6.x running HPE CIM providers, such as OEM HPE customized images.

Symtomps
Unable to power on new VMs, vMotion fails.
vkernel.log shows the following errors:
Cannot create file /var/run/sfcb/52494bef-1566-c7e5-6604-676ddd5b9c46 for process sfcb-CIMXML-Pro because the inode table of its ramdisk (root) is full.

You see alot of files inside /var/run/sfcb directory
sfcb_2
sfcb_4

Below you will find workarounds to address this issue.
Continue reading

Posted in VMware | Tagged , , , , , , | Leave a comment

Alert! Alerte! Achtung! Critical bug in vSphere 6

Backing up VMs in vSphere 6 can cause data loss in your backups! Earlier I wrote about critical bug with vSphere 5.5 update 3 related to backups which was absolutely unacceptable and here we go again…

Here’s symptoms from VMware’s KB:

When running virtual machine backups which utilize Changed Block Tracking (CBT) in ESXi 6.0, you experience these symptoms:
The CBT API call QueryDiskChangedAreas() API call can sometimes return incorrect changed sectors, which results in inconsistent incremental virtual machine backups.
Inconsistent virtual machine backups

Of course, no fix yet but let’s take a look at the joke of a workarounds they suggest:
– downgrade ESXi to version 5.5 and change VM hardware version to 10
– Shutdown VM before doing incremental backup
– Do full backup daily instead of incremental
Really? Do you think any of these solution are applicable in production environment? Ha ha…

VMware’s KB 2136854
I honestly feel horrible for IT Professionals caught by poor QA from VMware, yet again.

Update: 11/26/2015: VMware released patch to fix it: ESXi600-201511001

Posted in VMware | Tagged , , , | Leave a comment

3PAR WSAPI via Powershell

Earlier I’ve demonstrated how to use 3PAR CLI with Powershell. In this example I will show how to work with 3PAR’s WSAPI via Powershell and poll last time remote copy group was synchronized (i use Last Sync date from first volume in the remote copy group).

I was asked to create solution to monitor replication via Recovery Manager for SQL as it sometimes fails for whichever reason and we don’t get notification that our SQL server wasn’t synchronizing to DR side for a while. I have special place for this product from HP (read my earlier posts).

If you’re using WSAPI to only read information, i recommend you create brand new account with limited privileges, as oppose to using 3paradm.

Continue reading

Posted in 3par, Powershell, Random stuff | Tagged , , , , | 5 Comments