Tag: Server 2016

Deploying and Configuring Storage Spaces Direct (S2D)

This blog post will focus on deploying Storage Spaces Direct (S2D) with Windows Server 2016 (steps with Server 2019 should be very-very similar, if not exact…) in a RoBo (Remote Office Branch Office) configuration with Dell Ready Nodes (S2DRN) leveraging RDMA (Remote Direct Memory Access). Now that is a mouthful, so let’s focus on what is Storage Spaces Direct first.

What is Storage Spaces Direct? With Server 2016, Microsoft introduced Storage Spaces Direct (S2D) with the release of Server 2016. S2D allows you to take industry-standard servers and leverage the internal local drives within the nodes and create a highly-available, highly-scalable software defined storage. Using hyper-converged or converged architecture, you are able to quickly deploy, scale storage, while implementing features such as storage tiers, caching, all while taking advantage of RDMA networking.

What is RDMA? Remote Direct Memory Access, or in short, RDMA, is an enterprise networking technology that allows you to exchange data through memory, without consuming the CPU or Operating System kernel. RDMA allows your applications to have high IOPS and with very low latency, while leveraging either RoCe (RDMA over Converged Ethernet) or iWARP (Internet Wide Area RDMA Protocol).

Note: the steps below focus on a single node of a 2-node cluster. All the steps below need to be executed on the secondary node.


Network Connectivity

Before we begin implementing, deploying and configuring we need to plan out the networking connectivity design. However before we do that, we need to understand what our design will look like. Below is a high-level diagram that illustrates the network connectivity for the host management and VM traffic, and the RDMA (Storage) traffic.


Network Configuration

Next we should map out our IP configuration. With this 2-node deployment we know we need the following network adapters and the following IPs.

Traffic Class Purpose Minimum IPs required VLAN ID Tagged/Untagged IP Address Space VLAN IP Address
Out of Band (iDRAC) Remote Management 2 Untagged /29
Management (Host) Management of Cluster and Cluster Nodes 3 Tagged/Untagged /29
Storage 01 SMB Traffic 2 Tagged/Untagged /29
Storage 02 SMB Traffic 2 Tagged/Untagged /29

Now that we have defined our networking configuration, we can move forward with booting the nodes, and making some necessary changes to the BIOS.


BIOS Configuration

Launch the node, and log into the BIOS (usually F2 at the Dell prompt)… Next go to the Device settings and let’s configure the RDMA/QLogic adapters.

Your configuration should look similar to this. In my instance, I am leveraging iWARP and not RoCE. By default, the adapters will allow for both modes, but we want to force iWARP only.

Disable Virtualization Mode

Disable DCBX (Data Center Bridging)

  • Link Speed: SmartAN
  • NIC + RDMA Mode: Enabled
  • RDMA Operation Mode: iWARP
  • Virtual LAN ID: 1 (which is default)

Remember, this needs to be done to both RDMA adapters!!! Once the settings have been applied, and saved, go ahead and reboot the node. Remember to do the second node too!


Install & Update Operating System

Next, we now need to install the Operating System. As best practice, once the OS is installed, update the OS and update all network drivers.


Validate & Rename Network Adapters

Also, it is a good idea to rename the Network adapters. Before we do that, let’s just confirm the adapters are there and look right.

Get-NetAdapter


Install Windows Features & Roles

Once the OS has been installed, and patched. Next we now need to install the necessary roles and features, ie. Hyper-V, Failover Manager, etc.

Install-WindowsFeature -Name Hyper-V, Failover-Clustering -IncludeAllSubFeature -IncludeManagementTools -Verbose -Restart

Configure Host Network

Now we need to configure the host management network. In this step we will create a SET switch (Switch Embedded Teaming). This switch will not only team the two network (host) adapters but at the same time a SET switch will be created that will be leveraged by the guest VMs via Hyper-V.

New-VMSwitch -Name S2DSwitch -AllowManagementOS 0 -NetAdapterName 'NIC1','NIC2' -MinimumBandwidthMode Weight -Verbose

Within this code, note, NIC1 and NIC2 are the host management adapters that were renamed to make life easier.

Now we need to create and configure the host management adapter. We will do this by executing the following cmdlet. Please note, in my environment, the Host Management network is untagged.

Add-VMNetworkAdapter -ManagementOS -Name 'Management' -SwitchName S2DSwitch -Passthru | Set-VMNetworkAdapterVlan -Untagged –Verbose

Once we execute this command, and run the Get-NetAdapter cmdlet, we can now see we have an additional network adapter.

In the event you need to tag your Management adapters you can use the following cmdlet below as reference.

Set-NetAdapterAdvancedProperty -Name 'SLOT 3 PORT 1' -DisplayName 'VLAN ID' -DisplayValue 103 -Verbose
Set-NetAdapterAdvancedProperty -Name 'SLOT 3 PORT 2' -DisplayName 'VLAN ID' -DisplayValue 104 -Verbose

Great, now we can add the nodes to the domain, and set the Management network adapters with static IPs.


Create the Cluster, Configure Witness, Enable Storage Spaces Direct

Now that are nodes are domain joined, and static IPs have been applied to the host management network, we can now begin creating the cluster.

In the code below, I am going to create the cluster; add the two nodes to the cluster; provision the Quorum witness (file witness) and enable Storage Spaces Direct on the cluster.

$cluster="Cluster_Name"
New-Cluster -name $cluster -Node "node01", "node02" -StaticAddress "IP Address" -NoStorage -Verbose
#assign cluster quorum
Set-ClusterQuorum -Cluster $cluster -FileShareWitness "\\server\filewitness\UNCPatch"
#enable storage spaces direct
Enable-ClusterS2D -Verbose

Once we have executed the commands above, if we launch Failover Manager, we can now see the created Cluster, with the 2 nodes, and Storage Spaces Direct enabled.


If we go into the Pool, we can also now see our Software Defined Storage Pool. We now can create volumes off of this pool.

If we go into the Enclosures, we can now also see all the disks available within the nodes and all disks that are members of the Storage Pool.

Great, now we need to do some configuration on the RDMA Adapters… Also to note, in this scenario I have leveraged a file share witness for the cluster. I would highly recommend considering or using Azure Cloud Witness. The egress traffic is next to 0, and you can connect several clusters to the storage account. For more information, see the following blog post(s): HERE.


Change RDMA mode to iWARP on QLogic Adapters

Again, remember which RDMA adapter is which. As mentioned previously, I renamed all of the network adapters to keep things simple and easy to remember.

Set-NetAdapterAdvancedProperty -Name 'SLOT 3 PORT 1' -DisplayName 'RDMA Mode' -DisplayValue 'iWarp'
Set-NetAdapterAdvancedProperty -Name 'SLOT 3 PORT 2' -DisplayName 'RDMA Mode' -DisplayValue 'iWarp'

Now we can leverage the QLogic adapters with RDMA via iWARP for our Storage traffic.


Create Cluster Shared Volumes (CSV)

Now that our cluster is created, nodes have been added, RDMA is configured, we can now create a CSV that will be leveraged by the VMs as their data store. We will do this by creating the CSV with the following cmdlet.

New-Volume -StoragePoolFriendlyName "Storage Pool" -FriendlyName "Volume01" -FileSystem CSVFS_ReFS -size 2TB

Now I elected to keep the CSV small with a 2TB volume, however I did have another 3TB to work with.


Update Live Migration

We are almost there, we now need to update the Live Migration network. This will ensure we make use of the RDMA network and not the Management network. We will do this via Failover Manager console.

Also a good idea to rename the networks. As you can see, I have renamed my storage networks to Storage1 and Storage2, and the host management network to Management.

Go to the Failover Manager Console >> Right Click Networks >> Select Live Migration Settings >> deselect the Management network.

\

You may have also noticed, I have configured the networks and their cluster use. Storage networks will be only available for the cluster, and the Management network will be available for both the cluster and client (guest VMs).


Next steps

We have now successfully created a Storage Spaces Direct cluster, leveraging RDMA networking and using the iWARP protocol. We now also created a SET switch that can be leveraged by our VMs as their network adapter. We have now also created a Storage Pool, with a volume dedicated for our VM disks leveraging the Cluster Shared Volume.

Next steps is now to create a VM and leveraging Storage Spaces Direct!

Advertisements

Deploy an Azure Cloud Witness for your Failover Cluster Quorum for Windows Server 2016 & 2019 with PowerShell

For the longest time, when deploying a cluster with Windows Server, you only had the two options,

  1. Using a dedicated disk for the quorum, or
  2. Configuring an SMB file-share as the quorum witness

With Server 2016 and 2019, there is now a third option, Cloud Witness. The Cloud Witness leverages Azure Blob storage to provide that additional cluster/quorum vote.

Before showing you how this is done, one should understand the purpose of a witness/quorum is with respect to a failover cluster.

When one or more members of a cluster stops reporting to the other cluster members, there is a vote. The vote ensures that there is no split-vote, and ensures the cluster has a true owner. For example, in a two node cluster, if each node believe it is the owner, then this will cause a “split-brain”. In short, neither node will ever agree it is the owner (or not). This is where a quorum is required to determine who is the owner by providing the third vote, ie. majority. This ensures the cluster has a true owner by having the majority of votes. Each member gets a vote, plus the quorum.

Why this matters, in the even there is no quorum, a node from the cluster can be evicted and as a result will suspend all application services to prevent data corruption by more than one system writing data without the cluster services coordinating data writes and access. Depending on policies, VMs running on the ejected cluster member will either suspend operations or be migrated to other nodes before being ejected.

Below is a step-by-step guide on how to configure the Azure Blob storage as the Cloud Witness.

Assumptions:

  • The Azure Blob storage account has already been created,
  • The cluster with at least 2 nodes already exists.

Launch the PowerShell console as Administrator, and execute the following cmdlet:

Set-ClusterQuorum -CloudWitness -AccountName "storage_account_name" -AccessKey "primary_access_key"

Now if we go back to the Failover Manager console we can see we have successfully configured cluster with a Cloud Witness.

In conclusion, deploying a Cloud Witness for a Failover Cluster is very simple, and in case of power outage in one datacenter, maintenance on a node, etc. then the entire cluster and its members (nodes) are all given an equal opportunity. Not only is it recommended and a requirement for 2-node clusters, but for any number of nodes, having a quorum is key ensuring high-availability.  As mentioned, there are the traditional options such as using a dedicated disk or a file-share (SMB) as the cluster witness. However with Azure Blob storage with its 16×9 uptime, we can always ensure the quorum witness is online and available.

Deploy an Azure Cloud Witness for your Failover Cluster Quorum for Windows Server 2016 & 2019

For the longest time, when deploying a cluster with Windows Server, you only had the two options,

  1. Using a dedicated disk for the quorum, or
  2. Configuring an SMB file-share as the quorum witness

With Server 2016 and 2019, there is now a third option, Cloud Witness. The Cloud Witness leverages Azure Blob storage to provide that additional cluster/quorum vote.

Before showing you how this is done, one should understand the purpose of a witness/quorum is with respect to a failover cluster.

When one or more members of a cluster stops reporting to the other cluster members, there is a vote. The vote ensures that there is no split-vote, and ensures the cluster has a true owner. For example, in a two node cluster, if each node believe it is the owner, then this will cause a “split-brain”. In short, neither node will ever agree it is the owner (or not). This is where a quorum is required to determine who is the owner by providing the third vote, ie. majority. This ensures the cluster has a true owner by having the majority of votes. Each member gets a vote, plus the quorum.

Why this matters, in the even there is no quorum, a node from the cluster can be evicted and as a result will suspend all application services to prevent data corruption by more than one system writing data without the cluster services coordinating data writes and access. Depending on policies, VMs running on the ejected cluster member will either suspend operations or be migrated to other nodes before being ejected.

Below is a step-by-step guide on how to configure the Azure Blob storage as the Cloud Witness.

Assumptions:

  • The Azure Blob storage account has already been created,
  • The cluster with at least 2 nodes already exists.

Launching the Failover Manager within Windows Server manager, connect to the cluster, and do the following. Right click the cluster object and select More Actions > Configure Cluster Quorum Settings…

Next select the Advanced Quorum configuration..

Ensure we have all the nodes selected, as seen below.

Next, select the Configure a Cloud Witness:

Now we need to get our Azure Blob storage account name, and its primary account key. This can be retrieved from the Azure portal.

Now validate the settings and complete the configuration.

Now if we go back to the Failover Manager console we can see we have successfully configured cluster with a Cloud Witness.

In conclusion, deploying a Cloud Witness for a Failover Cluster is very simple, and in case of power outage in one datacenter, maintenance on a node, etc. then the entire cluster and its members (nodes) are all given an equal opportunity. Not only is it recommended and a requirement for 2-node clusters, but for any number of nodes, having a quorum is key ensuring high-availability.  As mentioned, there are the traditional options such as using a dedicated disk or a file-share (SMB) as the cluster witness. However with Azure Blob storage with its 16×9 uptime, we can always ensure the quorum witness is online and available.

Step-by-Step: Setup and Configure Azure Site Recovery (ASR) with Windows Server 2016 Hyper-V using ARM

Not too long ago, Microsoft announced the support of Windows 2016 and Azure Site Recovery (ASR). Microsoft’s announcement can be found HERE.

With that said, I decided to setup ASR with my Hyper-V 2016 environment. Rather than the typical blog posts (screenshots etc.,) I decided to create a step-by-step video that demonstrates how to setup ASR with Windows Server 2016 and Hyper-V. That video can be found HERE at Channel 9.

In addition this post is a series of blog posts for Azure Site Recovery (ASR).

Installing SQL 2016 for System Center Operations Manager (SCOM) 2016 – Step-by-Step

The following is a guide on how to install SQL 2016 for your System Center Operations Manager (SCOM) 2016 environment. I will be installing SQL 2016 on a brand-new server with Windows Server 2016 installed.


To begin, I am going to set the following accounts as a Local Administrator on the server. Also, I am going to be creating two SQL instances, one for the Operations database, and the other for the Data Warehouse. Since this is for my personal lab, I am not dedicated storage/drives for the databases.

 

Domain\Account Description
domain\SCOM_AA SCOM Action Account
domain\SCOM_DA SCOM Data Access/SDK Account
domain\SCOM_SQL_READ SCOM SQL Reader
domain\SCOM_SQL_WRITE SCOM SQL Writer
domain\SQL_SA SQL Service Account

1

Next, let’s run the setup wizard as the SQL_SA account to make life easier down the road…

2

First thing I noticed, between SQL 2012/2014 and SQL 2016, a few changes/features have been removed/added. One that stands out is, the SQL Server Management Studio (SSMS) console is no longer here. Hmm.. I guess we can always connect to the databases from a console on another server/PC.

3

As mentioned, I am dedicated an instance for the Operations DB, and one for the Date Warehouse DB.

4

Setting the  SQL Server Agent to Automatic, and specifying the service accounts for the two services.

5

Keeping the database engine collation as default, “SQL_Latin1_General_CP1_CI_AS“.

6

Here, I am adding all the SCOM/SQL service accounts and SQL service accounts as SQL server administrators.

7

Nice! This is new for SQL 2016 — being able to create TempDB‘s. Since my VM has 8 vCPU’s, looks like SQL 2016 picked up on that, and has decided to create a one-to-one relationship. Great, let’s get started within the installation…

8

Perfect! No errors. Keep in mind, we will need to repeat these steps to create the Data Warehouse instance.

15

16

Great! Now we can go ahead with the SCOM 2016 installation! See HERE, for that post.

If you need to install the SQL Server Management Studio (SSMS), continue reading…

(more…)

Step-by-Step: Setup and Configure Azure Site Recovery (ASR) for On-Premises Virtual Machine with Azure Resource Manager (ARM)

This post is a series of blog posts for Azure Site Recovery (ASR).

Here is a step by step walk-through on how to go about setting up and configuring ASR (Azure Site Recovery) and backing up your On-Premises Virtual Machines (VMs) with Azure Resource Manager (ARM).

First things, first, Azure’s Recovery Service Vault is a unified vault/resource that allows you to manage your backup and data disaster recovery needs within Azure. For example, if you are hosting your VMs on-premises you can create a link between your on-prem site and Azure to allow your VMs to be backed-up into Azure. This is regardless of your hypervisor, it can be either ESX or Hyper-V, either will work. However for the interest of this blog post, I will be setting up ASR for VMs being hosted on your On-Premises environment on a Hyper-V 2012R2 environment.



Configuring Azure

Step 1: Create a Recovery Services Vault

Within Azure Resource Manager (ARM), if we select New, within the Marketplace, select Monitoring + management, then select Backup and Site Recovery (OMS) within the featured apps. Of course if this is no longer present, just search for it within the marketplace.

1

Next we will now need to create our vault.

Give it a meaningful name, and you can either create a new Resource Group, or use an existing. I opted with existing, as I will (another post) next setup a Site-to-Site ASR.

2

Give this a few seconds, maybe minutes to do its thing…

Great, now our Vault is up and ready to go!

3

Step 2: Choose your Protection Goal(s)

Click Settings > Site Recovery (Under Getting Stated) > Step 1: Prepare Infrastructure > Protection Goal > And specify the following > Click OK:

  • Replicating to: Azure
  • Machines Virtualized: Yes, with Hyper-V
  • Using SCVMM (Virtual Machine Manager): No

4

Step 3: Setup the Source Environment

Next, we will now need to give our Hyper-V site a name, “Ravi-OnPrem” makes sense here, but give it something meaningful.

5

6

Once validated, we can now go ahead with the Azure Backup Agent. Download the Azure Backup Agent, and also, download the Backup Credentials.

7

Download the Agent and Credentials to the server you will be backing up. In my example, I will be backing up a Windows Server 2016 (RTM).

Step 4: Microsoft Azure Recovery Site (MARS) Agent Install

The Microsoft Azure Recovery Site (MARS) Agent is a pretty simple install, but here is what I experienced when installing:

1

2

Since my environment is pretty open, ie. No Proxy, no changes required here.

3

Your call here..

4

All good with the MARS prerequisites… Hit Install!

5

All good, time to register our server to our Recovery Services Vault.

 

Step 5: Register Server to Azure Recovery Services Vault

6

Here is where we will need that VaultCrentials file.. I hope you downloaded it as mentioned earlier… As you can see, back in the first few steps, when we created our Vault, the settings are now automatically inputted.

7

Here, I decided to let the wizard generate the Passphrase. I then saved the key locally to the server.

 

8

Perfect! Now we can go ahead and with the Azure Back: Site Recovery/Backup Schedule, etc.

Step 6: Configuring Microsoft Azure Backup

Going back to our On-Prem server, which by the way is a Windows 2016 OS, let’s launch Microsoft Azure Backup

Click on Schedule Backup within the (Right) Actions Pane:

1

Since this is a basic server, I only allocated 1 drive for this example, once we hit Backup, I am presented with the available drives.

2

Now we can begin defining our Backup Schedule

Step 7: Specify Backup Schedule

3

For this example, I want to back up the following server with the following properties:

  • Backup once a week @ 4AM, every Monday

Retention Policy will be as follows, see below:

4

Once you are satisfied with the policy, go ahead and hit next. Since we want to back up to Azure, and not an offline backup, we will backup over the network.

5

Have a look over before we do the initial backup.

6

Step 7: Initiate Backup Now

Going back to the main console, within the right pane, within Actions, let’s initiate our Back Up Now.

7

If we now double click within the job, we can see the Backup has begun….

8

Step 8: Validate Backup

If we go back to Azure, and take a look at our Vault properties, we can see there is a Backup in progress.

9

If we drill down within the Backup, we can see our server being backed-up.

10

After a few minutes, we can go back to the server, and track its progress:

11

 

And likewise, if we go within to the Azure Resource Manager, and within the Vault Backup jobs, and take a look at the details, we can see data is being updated to Azure.

12

 

Perfect!