Category: Virtualization

How to upload Custom Images to Microsoft Azure using PowerShell

In this post, I am going to show how to upload a custom image used in Windows Hyper-V (2016) to Azure cloud. I will be using a combination of the UI in Hyper-V and PowerShell in Azure Resource Manager. I will be working with Azure Resource Manager (ARM) and with Hyper-V 2016 with a custom image of Windows Server 2008 R2 SP1.

Okay, let’s get started.

Prepare On-Premises Virtual Machine Image

First, we need an image to work with. As mentioned, I am using a Windows Server 2008 R2 SP1 (yes, 2008 — needed it for a customer). The VM is Generation 1, which is not only a requirement for Windows 2008, but also a requirement for Azure, as it currently does not support Generation 2 VMs. See HERE to read more on preparing a Windows VHD.

Next, we need to install Hyper-V role on the VM. Since this is a nested VM, we will first need to enable nested-virtualization on the Hyper 2016 box. See a previous post on how to go about this HERE. Once that is complete, go ahead and install the Hyper-V role.

Next, we now need to SysPrep our VM. From an Administrative command prompt, navigate to %windir%\system32\sysprep and then execute the command “sysprep.exe”. Here, we will be using OOBE and enabling “Generalize”, also “Shutdown” the VM once SysPrep completes.

Once the VM is SysPrep’ed, we now need to compact the VHDx (remember Hyper-V 2016 here) and also will need to convert the VHDx to a VHD. This is due to the limitation of Azure at the moment, as it only supports Gen1 VMs and VHD’s.

Go into Hyper-V and within the VM properties, edit the Virtual hard disk. Then we will need to compact the virtual hard disk. Go ahead and do that..

Great, now we need to convert the VHDx to a VHD. Time for PowerShell!

Convert-VHD –Path “<source VHDX path>" –DestinationPath "<destination VHD path>" -VHDType Fixed -Verbose


Let this run (I let it go over night.. it was getting late =) )

Great, now we are ready to move on to Azure and more PowerShell.

Build Azure Container and Upload Image to Azure

First, we need to download  and install the latest AzureRM bits module locally to the Hyper-V box (if you have done this.. jump down a few lines…)

Install-Module AzureRM -Force

Next, since there was a recent update to the AzureRm module, I now need to update the module path location.

$env:PSModulePath = $env:PSModulePath + "; C:\Program Files\WindowsPowerShell\Modules"

Next, we will need to import the AzureRm module.

Import-Module AzureRM -Force

Next, we’ll need to log-in into our Azure account, and specify the subscription to want to work with. In my case, there are multiple Azure subscriptions tied to my email.

Login-AzureRmAccount
Get-AzureRmSubscription
#select the subsciption you will be working with -- if you have one, you can skip this line
Select-AzureRmSubscription -SubscriptionId "<ID>"

Next, we will create a resource group and storage account, and bind the account the group.

New-AzureRmResourceGroup -Name "ResourceGroupName" -Location "Canada East"
New-AzureRmStorageAccount -ResourceGroupName "ResourceGroupName" -Name "StorageAccountName" -Location "Canada East" -SkuName "Standard_LRS" -Kind "Storage"

If you want to change the storage type, to let’s say Geo-redundant, here are the other types of storage:

Valid values for -SkuName are:

  • Standard_LRS – Locally redundant storage.
  • Standard_ZRS – Zone redundant storage.
  • Standard_GRS – Geo redundant storage.
  • Standard_RAGRS – Read access geo redundant storage.
  • Premium_LRS – Premium locally redundant storage.

Now, we need to create a Container and grab the URL needed to upload our image. I did this through the Azure Resource Manager (ARM) Portal since I couldn’t figure out the PowerShell cmdlet (Get-AzureStorageBlob) — if you can get this to work, please let me know!

You can get the URL from the Web UI when you go into the Storage Account >> Blobs >> Container (in my case, I called it “VHD”) >> Properties.

Now we are ready to upload our image/VHD to Azure! For me this took about 2 hours, uploading a 80GB file @ 9-10MBs.

$rgName = "ResourceGroupName"
$AzureVHDURL = "URL"
$LocalVHDPath = "LocalPathtoVHD"
Add-AzureRmVhd -ResourceGroupName $rgName -Destination $AzureVHDURL -LocalFilePath $LocalVHDPath

Great, now we just need to register the VHD disk to the Gallery, and we can begin creating machines based off our image that is now in the cloud! — Another post! 🙂

Advertisements

Issues with Azure Active Directory and Login-AzureRmAccount

If you’re like me, you have probably banged your head against the wall a few times with the Login-AzureRmAccount cmdlet… I reached out to the Azure Development team and not only is this a known issue, but there is currently no solution at the time…. Hmm.

Here is a bit of the background story, followed with the problem and solution to the issue.

Background:

Using PowerShell to script an auto-login to Azure, and start (and shutdown) Virtual Machines (yes, OMS Automation could help/solve this, but in this scenario my customer is currently not on-board with OMS). At any rate, the script is designed to capture some data on a on-premises server, if the threshold breaks, then begin starting resources in Azure, likewise, if the threshold falls back then shutdown those same resources in Azure.

Problem:

Running the following code, I keep getting the a null entry for SubscriptionId and SubscriptionName. Even though the user I have created is a co-administrator and has access to all the resources necessary. Assuming the login did work and the data isn’t needed…when try to start my Azure VM I get an Azure subscription error. So, let me check the subscription details. Well, there we go, I get the following response, “WARNING: Unable to acquire token for tenant ‘Common’” ….. So what gives?

powershell-reply-1

powershell-reply-2

I check and confirm the test-user is in-fact an administrator in ARM (Azure Resource Manager):

arm-portal-1

Solution:

Turns out, the user account created, not only needs to be created and added to the resources with Azure Resource Manager (ARM), but also needs to be assigned as an Administrator within Azure Classic Portal.

classic-portal-1

classic-portal-2

classic-portal-3

Once the test-user was added within the Classic Portal Administrators and set as Co-administrator, I could then get SubscriptionId and SubscriptionName info populate, and Get-AzureRmSubscription with proper details. Yay! (Still get that tenant ‘Common’ warning however…)

powershell-reply-3

Now I can go ahead with my script!

I hope this helps you as much as it helped me.

Step-by-Step: Setup and Configure Azure Site Recovery (ASR) with Windows Server 2016 Hyper-V using ARM

Not too long ago, Microsoft announced the support of Windows 2016 and Azure Site Recovery (ASR). Microsoft’s announcement can be found HERE.

With that said, I decided to setup ASR with my Hyper-V 2016 environment. Rather than the typical blog posts (screenshots etc.,) I decided to create a step-by-step video that demonstrates how to setup ASR with Windows Server 2016 and Hyper-V. That video can be found HERE at Channel 9.

In addition this post is a series of blog posts for Azure Site Recovery (ASR).

Step-by-Step: Setup and Configure Azure Site Recovery (ASR) for On-Premises Hyper-V Host with Azure Resource Manager (ARM)

This post is a series of blog posts for Azure Site Recovery (ASR).

Here is a step by step walk-through on how to go about setting up and configuring ASR (Azure Site Recovery) and backing up your On-Premises Virtual Machines (VMs) with Azure Resource Manager (ARM).

First things, first, Azure’s Recovery Service Vault is a unified vault/resource that allows you to manage your backup and data disaster recovery needs within Azure. For example, if you are hosting your VMs on-premises you can create a link between your on-prem site and Azure to allow your VMs to be backed-up into Azure. This is regardless of your hypervisor, it can be either ESX or Hyper-V, either will work. However for the interest of this blog post, I will be setting up ASR for a Hyper-V 2012R2 host.



Configuring Azure

Step 1: Create a Recovery Services Vault

Within Azure Resource Manager (ARM), if we select New, within the Marketplace, select Monitoring + management, then select Backup and Site Recovery (OMS) within the featured apps. Of course if this is no longer present, just search for it within the marketplace.

1

Next we will now need to create our vault.

Give it a meaningful name, and you can either create a new Resource Group, or use an existing. I opted with existing, as I will (another post) next setup a Site-to-Site ASR.

2

Give this a few seconds, maybe minutes to do its thing…

Great, now our Vault is up and ready to go!

3

Step 2: Choose your Protection Goal(s)

Click Settings > Site Recovery (Under Getting Stated) > Step 1: Prepare Infrastructure > Protection Goal > And specify the following > Click OK:

  • Replicating to: Azure
  • Machines Virtualized: Yes, with Hyper-V
  • Using SCVMM (Virtual Machine Manager): No

4

Step 3: Setup the Source Environment

Next, we will now need to Prepare our source give our Hyper-V site a name, “Ravi-OnPrem” makes sense here, but give it something meaningful.

5

6

Now we need to download the ASR Provider Installer, along with the Vault Registration Key.

set-source3

Step 4:  Install and Configure the ASR Provider on Hyper-V Host

1

2

3

4

5

This Hyper-V host is not behind any Proxy…

6

If we go back to Azure, we can now see our Hyper-V host populated.

7

Step 5: Create a Replication Policy

Within our Vault properties > Settings > Manage: Site Recovery Infrastructure > For Hyper-V Sites: Replication Policies > +Replication Policies

8

Step 6: Associate Hyper-V Site(s)

Next we will need to Associate our Hyper-V site:

9

10

Great! Now we can continue on with Step 3 (Target Environment) of Step 1 (Preparing Infrastructure).

Step 7: Create a Storage Account + Virtual Network

8

9

Within the Replication, we have a few options here. I left mine as default (GRS) Geo-Redundant.

Next, we need to create a Target Virtual Network:

11

Now we can go ahead and setup the replication settings:

Step 8: Setup Replication Settings

12

Since we create the Replication Policy beforehand, this auto-filled. Next we need to do some Capacity Planning. Since this is simply a walk-through example, I elected to skip this, but for a real-production environment, I would highly recommend doing this.

Here is a link to Microsoft’s Capacity Planner for Hyper-V Replica.

14

Hit OK, and now we are ready to to move on to Step 2 (Replication Application)

15

16

This all should have populated since we created our Storage account and Virtual Network just earlier… If not, add them.

Now Azure should have connected with our Hyper-V host, we can now see our VMs within our Hyper-V host. Here we now need to select which machines we will want to include within ASR. For simplicity and variety, I am going to select a domain controller and a Linux machine.

17

Now we need to configure the VMs properties:

18

Once we are good, we can go ahead and apply the Replication Policy to our VMs.

19

Once satisfied, go ahead and hit “Enable Replication“.

20

 

Lastly, Step 3, we now need to complete creating our Recovery Plan:

Step 9: Create Recovery Plan

21

Great! All done? Before we say all done, let’s go back to our Hyper-V host, and configure the Network/Throttling bandwidth.

Step 10: Network/Throttle Bandwidth

My Hyper-V host is not equipped with a GUI as I am using Windows 2012R2 Minimal Server, so navigate here to launch the Microsoft Azure Backup Agent, “C:\Program Files\Microsoft Azure Recovery Services Agent\bin\“. Launch, “wabadmin“.

23

In the Actions pane, select “Change Properties” >> Select the Throttling tab.

24

Change these settings as to your needs. I wanted to increase my non-work hours to 4MB, but looks like 1MB is the max.

Great! Since we already hit, enable replication, this process should have already started… Let’s go back to Azure:

If we take a look at the Vault > Settings > Protected Items > Replicated Items 

25

Once these VMs are 100% Synchronized, the next steps will be to simulate a fail over, both Test and Planned.

26

 

Step-by-Step: Setup and Configure Azure Site Recovery (ASR) for On-Premises Virtual Machine with Azure Resource Manager (ARM)

This post is a series of blog posts for Azure Site Recovery (ASR).

Here is a step by step walk-through on how to go about setting up and configuring ASR (Azure Site Recovery) and backing up your On-Premises Virtual Machines (VMs) with Azure Resource Manager (ARM).

First things, first, Azure’s Recovery Service Vault is a unified vault/resource that allows you to manage your backup and data disaster recovery needs within Azure. For example, if you are hosting your VMs on-premises you can create a link between your on-prem site and Azure to allow your VMs to be backed-up into Azure. This is regardless of your hypervisor, it can be either ESX or Hyper-V, either will work. However for the interest of this blog post, I will be setting up ASR for VMs being hosted on your On-Premises environment on a Hyper-V 2012R2 environment.



Configuring Azure

Step 1: Create a Recovery Services Vault

Within Azure Resource Manager (ARM), if we select New, within the Marketplace, select Monitoring + management, then select Backup and Site Recovery (OMS) within the featured apps. Of course if this is no longer present, just search for it within the marketplace.

1

Next we will now need to create our vault.

Give it a meaningful name, and you can either create a new Resource Group, or use an existing. I opted with existing, as I will (another post) next setup a Site-to-Site ASR.

2

Give this a few seconds, maybe minutes to do its thing…

Great, now our Vault is up and ready to go!

3

Step 2: Choose your Protection Goal(s)

Click Settings > Site Recovery (Under Getting Stated) > Step 1: Prepare Infrastructure > Protection Goal > And specify the following > Click OK:

  • Replicating to: Azure
  • Machines Virtualized: Yes, with Hyper-V
  • Using SCVMM (Virtual Machine Manager): No

4

Step 3: Setup the Source Environment

Next, we will now need to give our Hyper-V site a name, “Ravi-OnPrem” makes sense here, but give it something meaningful.

5

6

Once validated, we can now go ahead with the Azure Backup Agent. Download the Azure Backup Agent, and also, download the Backup Credentials.

7

Download the Agent and Credentials to the server you will be backing up. In my example, I will be backing up a Windows Server 2016 (RTM).

Step 4: Microsoft Azure Recovery Site (MARS) Agent Install

The Microsoft Azure Recovery Site (MARS) Agent is a pretty simple install, but here is what I experienced when installing:

1

2

Since my environment is pretty open, ie. No Proxy, no changes required here.

3

Your call here..

4

All good with the MARS prerequisites… Hit Install!

5

All good, time to register our server to our Recovery Services Vault.

 

Step 5: Register Server to Azure Recovery Services Vault

6

Here is where we will need that VaultCrentials file.. I hope you downloaded it as mentioned earlier… As you can see, back in the first few steps, when we created our Vault, the settings are now automatically inputted.

7

Here, I decided to let the wizard generate the Passphrase. I then saved the key locally to the server.

 

8

Perfect! Now we can go ahead and with the Azure Back: Site Recovery/Backup Schedule, etc.

Step 6: Configuring Microsoft Azure Backup

Going back to our On-Prem server, which by the way is a Windows 2016 OS, let’s launch Microsoft Azure Backup

Click on Schedule Backup within the (Right) Actions Pane:

1

Since this is a basic server, I only allocated 1 drive for this example, once we hit Backup, I am presented with the available drives.

2

Now we can begin defining our Backup Schedule

Step 7: Specify Backup Schedule

3

For this example, I want to back up the following server with the following properties:

  • Backup once a week @ 4AM, every Monday

Retention Policy will be as follows, see below:

4

Once you are satisfied with the policy, go ahead and hit next. Since we want to back up to Azure, and not an offline backup, we will backup over the network.

5

Have a look over before we do the initial backup.

6

Step 7: Initiate Backup Now

Going back to the main console, within the right pane, within Actions, let’s initiate our Back Up Now.

7

If we now double click within the job, we can see the Backup has begun….

8

Step 8: Validate Backup

If we go back to Azure, and take a look at our Vault properties, we can see there is a Backup in progress.

9

If we drill down within the Backup, we can see our server being backed-up.

10

After a few minutes, we can go back to the server, and track its progress:

11

 

And likewise, if we go within to the Azure Resource Manager, and within the Vault Backup jobs, and take a look at the details, we can see data is being updated to Azure.

12

 

Perfect!

Step-by-Step – Upgrading to SCOM 2016 from SCOM 2012 R2

So you’ve decided to take the jump and upgrade to System Center Operations Manager 2016? Assuming you are running a SCOM 2012R2 (UR11) environment, this post will be exactly what you need to have a successful SCOM 2012 R2 to SCOM 2016 upgrade. **Although the recommended path is to upgrade from UR9**

In my environment, here is the quick breakdown:

  • SCOM 2012 R2 environment has two Management Servers.
    • Handful of clients (Windows 2012R2) being monitored.
  • Both Management Servers live on a Windows 2012 R2 operating system.
    • Both Management Servers are running with UR 11 (Update Rollup) **Microsoft recommends upgrading the environment at latest version number minus one (latest – 1), so in this case, UR9 (UR10 was never issued for SCOM 2012R2)**.
  • The SQL environment is a SQL Server 2014 SP2 also running on Windows 2012 R2.
    • The Operations and Data Warehouse live on dedicated SQL instances, however reside on the same server.

Pre-Upgrade Tasks

Let’s get started!

First, we need to do some “Pre-Upgrade” Tasks. Follow these in order.

  1. Back up the Operations Manager Databases
  2. Review the Operations Manager Event Logs
  3. Cleanup the Database (ETL Table)
  4. Remove Agents from Pending Management
  5. Disable the Notification Subscriptions
  6. Stop the Services or Disable any Connectors
  7. Verify that the Operational Database Has More Than 50 Percent Free Space
  8. Back up the Operations Manager Databases

To Cleanup the ETL Tables, you will need to run the following script:

  • The following script will determine the number of rows that will (need) be deleted:

DECLARE @SubscriptionWatermark bigint = 0;

SELECT @SubscriptionWatermark = dbo.fn_GetEntityChangeLogGroomingWatermark();

Select COUNT (*)
FROM EntityTransactionLog ETL with(nolock)
WHERE NOT EXISTS (SELECT 1 FROM EntityChangeLog ECL with(nolock) WHERE ECL.EntityTransactionLogId = ETL.EntityTransactionLogId)
AND NOT EXISTS (SELECT 1 FROM RelatedEntityChangeLog RECL with(nolock) WHERE RECL.EntityTransactionLogId = ETL.EntityTransactionLogId)
AND EntityTransactionLogId < @SubscriptionWatermark;

etl-cleanup-1

  • Now we can go ahead and clean up the ETL table running the script below:

DECLARE @RowCount int = 1;
DECLARE @BatchSize int = 100000;
DECLARE @SubscriptionWatermark bigint = 0;
DECLARE @LastErr int;

SELECT @SubscriptionWatermark = dbo.fn_GetEntityChangeLogGroomingWatermark();
WHILE(@RowCount > 0)
BEGIN
DELETE TOP (@BatchSize) ETL
FROM EntityTransactionLog ETL
WHERE NOT EXISTS (SELECT 1 FROM EntityChangeLog ECL WHERE ECL.EntityTransactionLogId = ETL.EntityTransactionLogId)
AND NOT EXISTS (SELECT 1 FROM RelatedEntityChangeLog RECL WHERE RECL.EntityTransactionLogId = ETL.EntityTransactionLogId)
AND ETL.EntityTransactionLogId < @SubscriptionWatermark;

SELECT @LastErr = @@ERROR, @RowCount = @@ROWCOUNT;

END

etl-cleanup-2

Great! Let this run, which may take a few minutes, or hours depending on your environment..

Now we are ready to get started with the upgrade. (Don’t get to backup your databases (Report Server, Operations, Data Warehouse!!)

1


Upgrading to SCOM 2016

After you have downloaded the RTM image file, as you may have noticed, it is not an ISO file. So let’s extract the file contents locally…

2

Before moving on with the install, I stopped the following SCOM services on all of the Management Servers:

  1. Microsoft Monitoring Agent (healthservice)
  2. System Center Data Access Service (OMSDK)
  3. System Center Management Configuration (cshost)

Now we can run the installer (Run As Administrator or, SCOM Data Access/SDK account)

3

4

5

6

7

Sweet! Although this was expected, since no new changes were required for Windows Server 2012 R2.

8

I cannot stress how useful it is to use dedicated service accounts. Here input your Data Access/SDK account.

9

Once quick review before we begin the Upgrade..

10

Let this run… For me, the upgrade took around 50 minutes for the first Management Server..

11

Sweet! All good. Remember to install the license key before the 120 days are up.

Let’s launch the console just to make sure we are in all working order.

12

Great! Now we will need to repeat the process for the second/other Management servers…

Once complete, let’s upgrade our client agents to SCOM 2016 (v 8.0.10918.0)

13

At this time we can make use of the SCOM 2016 features, and update our out-dated Management Packs. Remember this feature really only works for Microsoft based Management Packs, ie. SQL, Windows Server, Client OS, etc.

14

 

I hope this helped! For additional information, and or upgrading other items such as Gateways, ACS (Audit Collection Services), etc. Please visit Microsoft’s guides HERE.

 

Lastly, it is highly recommended to upgrade to SCOM 2016 Update Rollup 1 (UR1). For that guide, please visit this LINK.

 

Happy SCOM’ing 2016!

(more…)

Step-by-Step: Setup and Configure Azure Site Recovery (ASR) Virtual Machines (VMs) in Azure with Azure Resource Manager (ARM)

This post is a series of blog posts for Azure Site Recovery (ASR).

  • ASR for VMs hosted On-Premises, coming soon…
  • ASR for Hyper-V hosted On-Premises, coming soon…
  • ASR for an ESXi hosted On-Premises, coming soon…

Here is a step by step walk-through on how to go about setting up and configuring ASR (Azure Site Recovery) and backing up your On-Premises Virtual Machines (VMs) with Azure Resource Manager (ARM).

First things, first, Azure’s Recovery Service Vault is a unified vault/resource that allows you to manage your backup and data disaster recovery needs within Azure. For example, if you are hosting your VMs on-premises you can create a link between your on-prem site and Azure to allow your VMs to be backed-up into Azure. This is regardless of your hypervisor, it can be either ESX or Hyper-V, either will work. However for the interest of this blog post, I will be setting up ASR for VMs hosted within Azure.


Configuring Azure

Step 1: Create a Recovery Services Vault

Within Azure Resource Manager (ARM), if we select New, within the Marketplace, select Monitoring + management, then select Backup and Site Recovery (OMS) within the featured apps. Of course if this is no longer present, just search for it within the marketplace.

1

Next we will now need to create our vault.

Give it a meaningful name, and you can either create a new Resource Group, or use an existing. I opted with existing, as I will (another post) next setup a Site-to-Site ASR.

2

Give this a few seconds, maybe minutes to do its thing…

Great, now our Vault is up and ready to go!

3

Step 2: Backup Goal/Target

Select +Backup, and let’s setup create a backup strategy:

4

5a

As mentioned, in this walk-through, we will be setting up ASR for our VMs within Azure. So, this workload will be running against our Azure environment, and we want to backup our VMs.

Step 3: Create a Backup Policy

Now we want to create a backup policy. You can chose the default, which I believe is a daily snap-shot and the retention is 30 days. This may be too aggressive, or too conservative. Nevertheless, let’s create our own.

6a

Give it a name “ASRBackup14Days“, for this example, I want to backup my VMs in the following manner:

  • Backup every day @ 2AM
  • Retain the daily backup of the VM for 2 weeks (14 days)
  • Retain the weekly backup of the VM for 2 weeks
  • Retain the monthly backup of the VM for 2 months (~60 days)
  • Also, begin this policy the first day of January 2016…

Of course these options are..optional, you only need to specify either the daily, weekly or monthly retention…

Once complete, we now need to select the VM(s) we would like to back-up.

7a

Select one, or select them all, but keep in mind, this could get costly $$$$, more VMs and more often the back-up frequency.

8

Step 4: Initial Backup

Great! Now, Enable backup. Now, if we go back to our ASR Vault, should see a job already in progress, as Azure already started the initial backup.

9

10

As you can see, the VM is being backed up now!

Step 5: On-Demand Backup

If you ever want to do an ad-hoc backup, just go back to the ASR Vault, within the Protected Items, select the VM(s) you are interested, and schedule an immediate backup.

11