Azure

Microsoft have announced the preview of Enterprise Cost Management for Azure, which is great news for Enterprise Agreement customers.

Until now gaining visibility of spend on an Azure Enterprise Agreement has been difficult to manage even when combined with Tags and Resource Groups.

It should also be noted that an Enterprise Agreement doesn’t provide spending limits (see offer details), quotas or even billing alerts (see prevent unexpected costs) so customers are often wary of migrating services to Microsoft Azure and/or providing access to their Azure Portals due to fear of being stung by large bills.

It is understandable that Microsoft do not want to ‘turn off’ customers workloads, however their could be a case for this in a development environment where a person leaves a ‘monster VM’ up and running of a month by mistake.

This is a step in the right direction, hopefully we will see billing alerts added in the not to distant future.

 

Over the past three months, I have been leading a delivery engagement which has meant that I’m not as up to speed as I perhaps should have been on the latest enhancements to Microsoft Azure.

With this in mind, I thought I would share with you , the feature enhancements over the past few months that have had the biggest impact to the customers I work with.

Azure Service Health (Preview)

Planned and unplanned maintenance events are always a hot topic when educating customers on the use of cloud for IaaS as it’s a paradigm shift from the on-premises operating model.

Rather than having an email letting you know that West Europe is going to be patched in the future or checking the Azure Status URL, Microsoft have rolled this up into Service Health.

In a nutshell this lets you know what ongoing issues in Azure services are impacting your resources, provides you with a PDF summary of the issue for problem management.

Read more here.

Azure VM Configuration Changes (Private Preview)

Let’s face it a significant proportion of operational outages are caused by people making changes without following the correct internal procedures.  To circumvent this, Microsoft have introduced Azure VM Configuration Changes which can track all Windows Services, Linus Daemons, Software by default.

Azure VM Configuration Changes also allows you to view changes in the last 30 minutes, hour, six hours, 7 days or 30 days so you can pinpoint when changes occurred to the VM.

See more here.

Azure Large Disks

One of the challenges around IaaS VMs was trying to fit existing file structures into or across multiple 1TB hard drives.  This caused a few challenges for customers who had to rework GPO’s or migrate data to enable the use of file services within Azure.

Another significant challenge was using Azure Site Recovery to protect a VM with a hard drive larger than 1TB.  To address both of these issues Microsoft have launchged 4TB for Azure IaaS VM’s,

See more here and here.

Azure Application Gateway

Security is always a hot topic when it comes to cloud and Microsoft has fixed the gap it had between DNS based Global Site Load Balancing using Traffic Manager and Azure Load Balancer which worked at Layer 4 (TCP/UDP).

Azure Application Gateway acts as a Web Application Firewall to protect from common web attacks such as SQL injection, cross site scripting and session hijacks.

Read more here.

Faster Azure VPN Gateway

When customers embark on their cloud journey, it normally starts with a Site to Site VPN whilst ExpressRoute is put in place.  A previous limiting factor with Site to Site VPN’s was the bandwidth limit and SLA.

Microsoft have resolved this by introducing a new series of VPN gateways appropriately titled VpnGw1, VpnGw2 and VpnGw3 which will provide an SLA of 99.95% with up to 1.25Gbsp throughput at the same cost as the previous gateways.

Read more here.

 

This is a guest blog post by one of my Cisco CCIE colleagues Adam Stuart on his view of Azure Network Watcher.

What is it?

Azure Network Watcher is a feature within Microsoft Azure to make consumption of network data/troubleshooting easier.

How much does it cost?

Free until August 1st.

Then

• 5GB free network logs p/m, with small overcharge for extra GB
• 1000 checks p/m, small overcharge per extra 1000 checks

Plus storage costs for log retention.

What Does It Do?

• Monitoring Topology – Shows a very basic network topology diagram, no further drill down is possible.  To exactly useful but better than nothing.

• Diagnostics – IP Flow Verify, simple packet trace function to test is a source/destination is allowed via a NSG policy.  Equivalent of packet trace in Cisco land.  Quite useful if you have lots of NSG.  Overall a good sanity check.

• Diagnostics – Next Hop, simple utility to verify next hop as per effective routing table.  This would be useful for a customer using Network Virtual Appliance (NVA) Firewalls and complex UDR.  It provides insight into the Azure routing service which is otherwise tricky to obtain.

• Diagnostics – Provides details of NVA specific to network interface of VM.  Not useful unless you have overlapping NSG on a NIC and subnet and wanted to see the result of an aggregate policy
• Diagnostics – Packet Capture, this is essentially an easier way to run tcpdump and get pcap files form virtual machines. Note, you need to install a VM extension* to get this to work see here.
• Logs – NSG Flow Logs, the equivalent of checking Access-list log on a normal firewall. This is the primary function that most customer will be after. To answer the question “is the firewall blocking it”? Enabled on a per NSG basis, logs to a container in blob storage, which you export as JSON format.  This is probably quite powerful, but the default output is not very accessible. JSON format logs require another parser to provide any real value.

When you shift your focus from on-premises architecture to cloud based services, you notice that the velocity of updates and new features is relentless.  To give you an idea, over the last 12 months Microsoft have released over 500 updates on the Azure platform, some of these are feature enhancements, betas, public previews and new services.

Microsoft Azure is a moving target keeping up to date with enhancements is a full time job!

With this in mind, I thought I would share with you , the feature enhancements over the past few months that have had the biggest impact to the customers I work with.

Azure DevTest Labs – Ability to set an expiry date on virtual machines

We all know that development environments should only be up and running for a short amount of time to facilitate an application enhancement or initiative.  However in reality the virtual machines which were only meant to be temporary, end up being on all the time with everyone afraid to power them off ‘just in case something bad happens’.

With Azure DevTest Labs we can now harness the inbuilt functionality to set expiry dates to virtual machines.  Sounds trivial but what a great feature addition to manage the compute resources around a VM lifecycle.

Read more here.

Azure Automation – Start/Stop VMs (Preview)

We are used to having our on-premises VMs running 24×7.   Because we have already invested in the compute and storage infrastructure so what is the real use case of shutting them down?  Instead we sweat our assets for the three to five year hardware lifecycle and rinse and repeat.`

Taking the same architecture principles to a cloud based platform increases costs.  Why not get smart and reduce your consumption costs by targeting applications to when users access them?  If HR only work Mon-Fri 9-5 and access Sage during this time frame, then why not power on the VM at 8am and shut it down at 6pm?  Reducing your Azure consumption costs by 58% over a 12 month period.

Read more here.

Azure Managed Disks

When running virtual machines in Azure it’s not just a simple case of creating a storage account, factors such as the number of virtual machines per storage account with their IOPS requirements as well as the impact that backups have need to be taken into consideration.

These manual considerations can be negated to a point using Azure Managed Disks in which Azure handles the Storage Account in the back end reducing your management overhead.

With the general availability release of Azure Managed Disks integrate with Azure Backup and also Disk Encryption.

Read more here.

Automated Backup for SQL Server 2014 and 2016 Virtual Machines

It can be common for enterprises to backup SQL Server databases outside of the general backup schedule applied to other applications.  With the release of Automated Backup for SQL Server 2014 and 2016 Virtual Machines this can be automated for you with the creation of the VM.

With a daily backup and a retention period of 30 days, this adds another layer of protection to your backup routine.

Read more here and here.

Azure Backup Instant File Recovery (Preview)

Azure Backup has always done a job, but backing up a VM level has had its disadvantages namely having to restore an entire VM to an alternate location to get back a single file or folder.

Azure Backup Instant File Recovery creates a writable mount point attached to the VM you want to restore data too using an iSCSI target.  This simplifies the process and reduces management overhead.

Read more here.

Azure Security Center Enhancements (Preview)

The Azure Cyber Security  team have announced a number of the new enhancements which are in preview.  The most poignant ones are:

• Application White Listing – Allow only the authorised executables to run within the virtual machine, with Azure Security Center discovering and recommending white listing policies
• Just in Time Network Access to VM’s – Reduce your attack surface by only allowing access to common ports when required

By simply deploying the above two changes, I can see a benefit for most organisations to enhance their security footprint.

Read more here.

When deploying workloads to the public cloud, the question arises ‘how do you monitor them’?  This then leads to further questions such as:

• Will my existing monitoring solution support Microsoft Azure workloads such as PaaS?
• Do I need to purchase extra licenses or upgrade my existing licenses?
• Do I need to have two different monitoring solutions? One for on-premises and one for the public cloud?

Each of the above questions then leads to a myriad of further questions around the deployment mechanism, how data is collected, stored and displayed.  How are you altered of issues or potential issues?  How do you capacity plan for resources in the cloud?  How do you monitoring specific application workloads?

To answer these questions and more Microsoft released Operations Management Suite which became generally available in January 2016.

What is Operations Management Suite?

Operations Management Suite is ‘Management as a Service’ or MaaS for short.  It runs in Microsoft Azure and can provide visibility into your on-premises and Microsoft Azure based workloads, providing a consistent monitoring approach across datacentres.

OMS is broken down into four key components which at a high level are:

• Insight and Analytics to collect, correlate, search and act on log and performance data generated by operating systems and applications. Providing real time analysis of information and potential issues.
• Automation & Control which enables a consistent approach to control and appliance by leveraging desired state configuration, change tracking and update management.
• Security and Compliance focuses on identifying, assessing and mitigate risks to infrastructure. Collecting and analysing security events to identify suspicious activity.
• Protection and Recovery to provide analysis and status updates of Azure Backup and Azure Site Recovery

The diagram below depicts a logical overview of the proposed Operations Management Suite environment.

Note: At the time of writing OMS supports Azure Backup and Site Recovery in Classic Mode.

Operations Management Suite Components

The components of Operations Management Suite are broken down into three areas, agent, dashboard and solution packs.

• Agent is an in-guest service which can be pushed out automatically using Group Policy, System Center Configuration Manager or another deployment method. It is used to provide heartbeats and data back to the centralised Operations Management data repository
• Dashboard is the Operations Management Suite portal which runs in a browser. The dashboard can be customised with graphical views of valuable searches and solutions
• Solution Packs are add-on services which add functionality and provide in-depth analysis of collected data. Examples of commonly deployed Solution Packs are:
  • Malware Assessment which provides status of antivirus and antimalware scans across servers
  • Change tracking with tracks configuration changes across servers
  • System Update Assessment which identifies missing system updates across servers
  • AD Replication Status which identifies Active Directory replication issues
  • SQL Assessment which assesses the risk and health of SQL Server environments
  • AD Assessment which asses the risk and health of Active Directory environment

Microsoft are continuously updating Solution Packs and a few which are in public preview are listed below:

• Azure Networking Analytics which enables you to gain insight into Network Security Groups and Application Gateway logs
• Capacity and Performance which enables you to view Hyper-V CPU, memory and storage utilisation
• Office 365 which provides visibility into user activities as well as forensics for audit and compliance purposes
• Network Performance Monitoring which offers real time monitoring of parameters such as loss and latency
• System Centre Operations Manager Assessment which asses the risk of your SCOM environment
• VMWare Monitoring provides the ability to explore ESXi Host logs for monitoring, deep analysis and trending

The graphic below provides an example Operations Management Suite dashboard.

Workspaces

OMS uses the concept of workspaces which is primarily an administrative boundary but is also used to collect data within an Azure region.  Workspaces can be used to delegate responsibility to individual users or groups who undertake specific roles e.g. Network Team access to Network Performance Monitor.

It should be noted that workspaces are independent of each other and that data collected from each workspace cannot be viewed in another workspace.  However you can link multiple workspaces to a single Microsoft Azure subscription.

Workspaces also enable the use of different license plans, for example in one workspace you might use the System Center Add On and another workspace you might use Insight & Analytics.

Data Collection

Operations Management Suite collects data on a real time basis using either in-guest agents installed on Windows or Linux, a System Center Operations Management Group which uses the SCOM management servers to forward events and performance data to Log Analytics or finally an Azure Storage Account that collects data from PaaS and IaaS services.

• Logging which is data generated by the operating system or application such as event logs, IIS logs, syslogs or custom logs in the form of text files.
• Performance which uses the Windows or Linux performance counters to collect data such as memory, processor and disk information
• Solution specific items which provide in-depth analysis of application items

A logical overview of data collection is shown below.

Licensing

OMS can be licensed either on a pay as you go basis or on a subscription basis.  You are given the choice of licensing all OMS components together as a ‘suite’ which makes the overall cost cheaper or you can pick which components you need.

• Licenses are based on nodes, a node is defined as a physical computer, virtual machine or network device
• Node charges are hourly and nodes that only report for a part of a month are proratared
• Each node can produce up to 500MB of data per day without incurring any extra charges
• OMS data retention is currently set to one month, plans to expand this to two years are in the pipeline

Final Thought

OMS is maturing as a product and integration points to on-premises environment is evolving.  The ability to provide a centralised dashboard with application or vendor specific solution packs will make the product more appealing.  Watch this space!