Brain Dump(ster fire)- POC Azure Automation, MEM/CM, On-Prem SQL, and Email

(I put out a Twitter poll asking if I should post a “nerdy” topic, or a “stupid AF” topic first, and nerdy was in the lead, so grab your caffeine. You’re going to need it)

UPDATE 1 – I forgot to mention a few things:

• You will need to install the PowerShell module dbatools on the Hybrid Worker machine.
• You don’t need to install the Hybrid Worker on CM01, but any machine in your lab environment, as long as the MECM machine and SQL database are accessible to it.
• You don’t technically need PowerShell module ImportExcel on the Hybrid Worker. If it’s installed, the script will export directly to a Excel worksheet (whether Excel is installed or not). If not, it exports to CSV format.

When I was a little boy, okay, littler boy, in the 1970’s, my favorite things in life were: G.I. Joe, bicycles, and Lego kits. Holy shit! 1970’s? Phone booths. Pre-cable TV. Wall-mounted phones. Gas was $1.35/gallon. And disco was the douche king of fashion nausea. Computers? Those were still science fiction stuff to us.

Anyhow, when Christmas morning came around, and I saw any one of those “magic 3”, you could’ve set fire to the entire world and I’d never noticed it. My focus was entirely on them. When it came to Lego kits, we had two (2) kinds to choose from: project kits and master kits…

Project kits were like buying a bicycle at Walmart. Master kits were like buying a Home Depot. Hopefully you get the idea. My favorite were the master kits, which I would just dump into a bigger pile with last-year’s kits and make some strange, but cool new thing. Frankenkits, I called them. I built cities with mono-rails, and spaceports, and ran the Christmas tree lights throughout. Half of the living room and then, well, someone would inevitably step on a brick with their bare feet, and alas, the Lego city was no more.

With software, one of my favorite things to work on are proof-of-concept projects. They’re like mini-projects, and mostly for myself, but are also at times part of a customer engagement, vetting a “could we do this?” question. Sometimes just to see if “idea 1” or “idea 2” was easier or better, or provided something over the other.

This post is going to be a simple, proof-of-concept, for setting up Azure Automation, with a Hybrid Worker, to invoke data exports from the SQL Server database underpinning a Microsoft Endpoint Configuration Manager (MEMCM) primary site, and sending the data via Email attachment.

A Little Background

It’s going to take years to shake the habit of referring to “System Center Configuration Manager” as MECM or MEMCM. The terms “Config-Manager”, “SCCM” and so on, are so ingrained in our minds that they’ll be tossed around long after I’m gone, I’m sure of it.

The test configuration I used for this is as follows:

• Windows Hyper-V host
• DC01
  • 1 gb memory
  • 2 processors
  • [C:] 80 gb disk (differencing on Windows Server 2019)
• CM01
  • 32 gb memory
  • 2 processors
  • [C:] 80 gb disk (differencing on Windows Server 2019)
  • [D:] 300 gb disk “Apps” (dynamic)
  • [E:] 300 gb disk “Data” (dynamic)
• FS01
  • 8 gb memory
  • 2 processors
  • [C:] 80 gb disk (differencing on Windows Server 2019)
  • [D:] 500 gb disk “Data” (dynamic)
• GW01
  • my router VM running pfSense 2.4.4
  • 512 mb memory
  • 1 processor
  • 50 gb disk (dynamic)
  • 2 NICs: WAN and LAN with LAN on 192.168.2.1 (internal Hyper-V switch)

This setup is only for testing purposes, not production. So allocated resources for VM guests are minimal.

In addition to the basic setup, the MECM site has to be operational (i.e. it has managed clients, inventory data, and has data flowing through its SQL veins). Otherwise, there’s nothing to query and nothing to report, so game over. If your site isn’t yet operational, pause here, get that resolved, and come back. I’ll wait.. .. . . …. . . . . .. . . . . .

Disclaimers:

• I am NOT an “expert” with Azure or Azure Automation. I may be using a bad approach somewhere along the way, but it’s just a proof-of-concept.
• The “cloud” is a moving target. Anything I post may likely change within a month or year, so this is all based on November 2019.
• DO NOT TEST any of these in production environments, Azure or on-prem, or I will disown you and plug my ears, while saying “la la la la la la laaaaa la la la” until you leave.
• This blabbering makes the assumption that you have an Azure subscription, with permissions to create Automation Accounts, and drain your bank account when you run crazy tasks and forget to check back that they aren’t spinning out of control somewhere.
• None of this is about “should you“, but simply intended to demonstrate ONE possible option. You may have a much better alternative, and that’s fine. Love, peace and good vibes to all. Namaste.

Now, before my grass-fed non-GMO brain gets cold…

Part 1 – Azure Automation Setup

Perform the following in the Azure portal. Technically, the SendGrid part isn’t part of the Automation Account, but it’s going to be needed for sending emails later on.

1. Create an Azure Automation account
   1. Search “Automation Accounts” and click to open
   1. Click “Add”
   2. Name: “aa-hybrid1“
   3. Subscription: <select yours>
   4. Resource Group: Create New / “rg-hybrid1“
   5. Location: <select the closest to you>
   6. Create Azure Run As Account: <yes>
   7. Create
2. Configure a SendGrid account
   1. Search “SendGrid Accounts” and click to open
   1. Click “Add”
   2. Name: “mailsender“
   3. Password: <make it diabolically complex>
   4. Confirm: <again>
   5. Subscription: <select yours>
   6. Resource Group: <select “rg-hybrid1” >
   7. Pricing: > choose “F1 Free“, review details, click Select
   8. Contact Information: > fill-out information, click OK
   9. Legal terms: > Review, click Create
   10. Click on the SendGrid account, then click Configurations. Copy the Username to Notepad for use later on.

Part 2 – Create Runbook Assets

Perform the following in the Azure portal. Runbook Assets are resources which support Runbooks, such as Variables, Credentials, Connections, Credentials, and so on.

Variables

1. Navigate to Automation Accounts and select “aa-hybrid1“
2. Scroll down and select “Variables“, click Add a Variable for each below:
   1. Name = “CM-SQL-HostName” Value = <FQDN of on-prem SQL host>
   2. Name = “CM-SQL-Database” / Value = <“CM_P01” or whatever>
   3. Name = “MailSender” / Value = <make up a sender address, it doesn’t have to actually exist>
   4. Name = “MailRecipients” / Value = <valid addresses to send email to>

Credentials

Scroll and select “Credentials“, click Add a Credential:

1. Name = “SendGridAccount“
   1. Copy the User name from SendGrid account properties, and enter the same password you used when setting it up, confirm the password again, and click Create
2. Name = “Contoso_CMINstall“
   1. Enter “contoso\cm-install” (or whatever your on-prem account happens to be) and enter the password, confirm the password again, and click Create

Part 3 – Create a Runbook

Perform the following in the Azure portal.

1. Within the aa-hybrid1 Automation Account section (okay, “blade”) scroll down and click on “Runbooks”
2. Click “Create a runbook“
   1. Name = “Export-CM-Devices“
   2. Type = “PowerShell“
   3. Description = “This is my runbook. There are many like it, but this one is mine.”
   4. Click Create
3. When the editor panel is ready, copy and paste the code from this link, then click Save (you can trust it, I think it works most of the time – lol)

At this point, you should have the following items created and configured. However, you still need to setup a Hybrid Worker in your on-prem environment, so you can run this Runbook “on” the on-prem server. Which we’ll do in part 4.

• Automation Account (with Resource Group)
• SendGrid Account
• Runbook Assets (Variables, Credentials, Run As Account)
• Runbook

Part 4 – Install a Hybrid Worker

Perform the following on your intended on-prem machine (virtual machine hopefully). For this demo use machine “CM01”, which is your MECM/ConfigMgr primary site server with SQL Server. We need to install a PowerShell script and edit some values before running it to create the new Hybrid Worker (HW) and Hybrid Worker Group (HWG)

A HWG is a logical name for one or more HW machines. When you initiate a Runbook for on-prem execution, it is targeted to the HWG, which in-turn hands it off to one of the HW’s on-prem. For this part we will create both an HW and HWG in one step.

1. Open a PowerShell console using “Run as Administrator“
2. Run Set-ExecutionPolicy ByPass
3. Run Install-Script -Name New-OnPremiseHybridWorker (answer “Y” to any confirmations. This script is published by Microsoft, a trusted name in quality software for over 500 years)
4. Run $cred = Get-Credential (enter your Azure account which has permissions to create Runbooks, etc.)
5. Type the following and press Enter. Remember to replace the “<…>” items with actual values for your Azure environment. Be patient after you press Enter, as it needs to download and install a few PowerShell modules, install some components, and entertain you with progress bars along the way.

$rg  = "<name of resource group>"
$aa  = "<automation account name>"
$sid = "<subscription id>"
$wg  = "mecmLab"
$cred = Get-Credential
New-OnPremiseHybridWorker.ps1 -AAResourceGroupName $rg `
    -SubscriptionID $sid -AutomationAccountName $aa `
    -HybridGroupName $wg -Verbose

Note: If you get tired of being prompted about an “Untrusted repository” when referencing PSGallery, type in Set-PSRepository -Name PSGallery -InstallationPolicy Trusted, and press Enter.

After the script processing has completed, review the output to confirm “Registration Complete” near the bottom, and you see the HWG name and machine name as well.

Then, go back to the Azure portal to complete the following:

1. Go to Automation Accounts / aa-hybrid1
2. Click “Hybrid worker groups” (down below Runbooks on the menu)
3. Verify you see the new HWG (e.g. “mecmlab”)
4. Click on the new HWG (e.g. “mecmlab”) to show more details
5. Click on “Hybrid Workers“
6. Verify you see your on-prem machine in the list (e.g. “CM01”)

Back on your on-prem machine (CM01) open Apps and Features (or Installed Applications list), and verify you see “Microsoft Monitoring Agent” installed and note the installation date.

You should now have everything configured and ready to rock-n-roll for running Azure Runbooks on-prem! Drink up! You might need it for the Normandy beach landing you’re about to enjoy. Just kidding.

Part 5 – Test your Hybrid Worker

For this step, I recommend a very basic/simple PowerShell Runbook to just display the computer name. That way you can see if it’s really executing on the (on-prem) Hybrid Worker, from within the Azure portal.

1. Create a New Runbook
   1. Name = “Show-ComputerName“
   2. Type = PowerShell
   3. Description = (blank)
   4. Click Create
   5. In the editor panel, type in the following code snippet:
      Write-Output "computer name: $env:COMPUTERNAME"
   6. Click Save
   7. Click “Test Pane“
   8. Change “Run Settings” option to “Hybrid Worker“
   9. Select the new HWG from the listbox (e.g. “mecmlab”)
   10. Click Start
   11. Wait for the output panel to return the results. You should see “Completed” at the top, and “computer name: CM01” below it.
   12. Up top, in the breadcrumb menu list, click on “Edit PowerShell Runbook“. If you are prompted about “Your unsaved edits will be discarded” click OK (they’re fine)
   13. Click Publish, and then click Yes

You have now reached “World Domination Level”, collected all of the treasures, and have crushed all your enemies. Grab another can of Red Bull and proceed quickly to level 6.

Part 6 – Run the Data Export Runbook

If you don’t already have a Runbook named “Export-CM-Devices”, go back to Part 3 and come back to this part when you’re ready. Perform the following in the Azure console, but keep an eye on your HW machine as well.

I recommend opening Task Manager within the HW machine and click the Details pane, so you can see what’s going on and what sort of latency you see between “Queued” and “Running” (in Azure) and what you see happen in the on-prem machine.

1. Navigate to Automation Accounts / Runbooks
2. Select the Runbook: Export-CM-Devices
3. Note: If the current Runbook status is still “In Edit”, click “Edit” and make sure you didn’t screw the code up while drinking your Red Bull with Bourbon last night. If it looks good, click Publish and click Yes.
4. Click Start
5. Switch Run Settings to Hybrid Worker, and select your Hybrid Worker Group from the listbox (e.g. “mecmlab”)
6. Click OK
7. Wait for the status to show “Running”, while watching Task Manager on the on-prem machine (e.g. CM01). You should see a new powershell.exe process pop up, note the time it takes from when Azure changes status from “Queued” to “Running” and when you see the powershell.exe process launch. Then note when it closes and when Azure shows “Completed” (or hopefully not “Failed”).
8. Click the Output tab and wait for output
9. If all goes well, you should see Status “Completed” and a message from output below showing “Sending an email to <whoever> :: subject = CM Inventory Report”
10. Check your Inbox for the email and attachment. Note that due to how you configured the “MailSender” address, the message may land in your Junk/Spam folder.
11. Open the attachment, gaze at the data, and consume your beverages now.

Ding! Fries are done!

Thoughts? Comments? Bugs? Post a comment below…

Happy Thanksgiving!