The services in Azure that are considered fundamental can be debated. A service such as Azure SQL Database may be considered fundamental to some, but not others. The fundamental services laid out in this book are those that the authors feel, after many years of real-world projects, to be fundamental to a wide majority of Azure users.
One of the joys of working with the Azure platform is the breadth of services it offers. There are many more services offered today than just a few years ago . . . or even one year ago when the first edition of this book was published! This section will briefly review several of the Azure services not covered in earlier chapters. It should be noted that this chapter does not provide an exhaustive listing of Azure services. The goal here is to provide exposure to other services available in the Azure platform. Some may be fundamental to you—today or in the future.
Azure Service Fabric represents the next generation hosting environment for cloud-scale solutions. Service Fabric makes it much easier to deploy and manage highly scalable, available, and reliable services. Applications designed as microservices are well suited to run on Service Fabric, primarily due to Service Fabric's runtime and lifecycle management capabilities—providing comprehensive capabilities for failover, leader election, state management, live upgrades with rollback, and automatic scale-up and scale-down.
Service Fabric is a proven platform for running critical, cloud-scale solutions. In fact, Microsoft uses Service Fabric to power some of Microsoft's most critical services, such as Skype for Business, Cortana, Azure SQL Database, Azure Event Hubs, and several more.
Service Fabric is not a platform that runs exclusively on Azure. You can deploy Service Fabric onpremises, in other cloud platforms, or, of course, in Azure. Microsoft has also committed to providing Service Fabric for both Windows and Linux operating systems. All this makes Service Fabric a platform you can run pretty much anywhere!
Cloud Services is a platform as a service (PaaS) compute feature in which applications are deployed into instances (managed virtual machines) of server types referred to as web roles and worker roles. The deployment of the instances is fully managed by Microsoft, making it easy to scale applications in and out.
After writing your application, you create a deployment package for your application and upload it to Azure; Azure will create the requested number of VMs and install your software on them. Azure manages the VMs, bringing up new instances if one crashes, and handling the updates without downtime. Azure also manages the load balancing and autoscaling for the VMs. You can easily change the number of VMs by modifying the instance count in the classic Azure portal. If you increase the instance count, Azure takes the original deployment package and deploys additional instances. If you decrease the instance count, Azure shuts down and removes instances.
Cloud Services web and worker roles are classic resources, and can only be used with the classic version of resources such as virtual networks and storage accounts. The Resource Manager deployment model is the recommended deployment model moving forward in Azure. For Cloud Services, the recommendation is to use other parts of Azure to run these kinds of workloads. New web workloads are better handled by the App Service feature. The worker roles that loop infinitely and do work such as reading messages from a queue can be migrated to WebJobs or the new Service Fabric product.
One of the hottest technology trends in the last two years is containerization, with the open source project, Docker, being one of the leading platforms for managing containers. Docker containers provide an efficient, lightweight approach to application deployment by allowing different components of the application to be deployed independently into different containers. Multiple containers can reside on a single machine, and containers can be moved between various machines. The extreme portability of the container makes it very easy for applications to be deployed in multiple environments—either on-premises or in the cloud—often with no changes to the application.
There is more to deploying and managing container applications than simply using Docker. Supporting services such as monitoring, clustering, orchestration, scheduling, and a way to visualize the environment are also needed. There is a rich, yet still maturing, ecosystem to help with these needs. This is where the Azure Container Service (ACS) provides value.
ACS supports Docker container images and simplifies the process of creating, configuring, and managing the VMs that provide the infrastructure for a Docker cluster. ACS includes industry-leading tooling from Apache Mesos-based DC/OS and Docker Swarm to provide an optimized configuration for resource orchestration. For workload scheduling, ACS includes the popular Marathon and Chronos frameworks. By providing an optimized configuration of open source tools and technologies, ACS makes it much quicker and easier for you to migrate container applications to and from Azure. You use an Azure Resource Manager template to deploy the cluster, selecting the size, number of host VMs, and orchestration tool of choice. Once the ACS cluster is deployed, you can use your existing management tools to connect and manage your container applications.
DocumentDB is a fully managed document database as a service designed to handle large amounts of data with no defined, rigid schema. It is highly available and performant, and it can be scaled up and down on demand. DocumentDB easily supports applications that need key value, document, or tabular data models.
A DocumentDB database is comprised of JSON documents. By default, all of these documents are indexed automatically, so you don't need to define secondary indexes for advanced searching. Because the data is schema-free, as your applications or their data requirements change, you don't need to make modifications to a schema to ensure your data models continue to work.
DocumentDB enables complex ad hoc queries using a SQL dialect. It is also integrated with JavaScript, allowing you to execute application logic within the database engine in a database transaction. This capability enables you to perform multidocument transaction processing using stored procedures, triggers, and user-defined functions.
Another interesting feature of DocumentDB is that it has protocol support for MongoDB. This means if you have applications written for MongoDB, you can use DocumentDB as the data store by installing some drivers and (in some cases) simply changing the connection string to point to the DocumentDB.
DocumentDB may look like other NoSQL database options out there, but the ability to perform transactional processing and complex queries make it particularly useful.
Redis is an open source, in-memory data structure store often used as a cache, database, or message broker. Azure Redis Cache is based on the popular open source Redis. The difference is that Azure manages Redis for you, saving you the trouble of spinning up a VM and installing and managing Redis yourself while still giving you a secure and dedicated Redis cache that can be accessed from any application within Azure. You can provision a Redis cache using the Azure portal.
Azure Redis Cache is available in three tiers:
Azure HDInsight is a fully managed Apache Hadoop service, using the Hortonworks Data Platform
(HDP) Hadoop distribution. HDInsight also includes other popular platforms that are commonly deployed alongside Hadoop, such as Apache HBase, Apache Storm, Apache Spark, and R Server for Hadoop.
HDInsight is used in big data scenarios. In this case, big data refers to a large volume of collected— and likely continually growing—data that is stored in a variety of unstructured or structured formats. This can include data from web logs, social networks, Internet of Things (IoT), or machine sensors— either historical or real-time. For such large amounts of data to be useful, you have to be able to ask the right question. To ask the right question, the data needs to be readily accessible, cleansed
(removing elements that may not be applicable to the context), analyzed, and presented. This is where HDInsight comes into the picture.
The Hadoop technology stack has become the de facto standard in big data analysis. The Hadoop ecosystem includes many tools—HBase, Storm, Pig, Hive, Oozie, and Ambari, just to name a few. You can certainly build your own custom Hadoop solution using Azure VMs. Or you can leverage the Azure platform, via HDInsight, to provision and manage one for you. You can even deploy HDInsight clusters on Windows or Linux. Provisioning Hadoop clusters with HDInsight can be a considerable time saver (versus manually doing the same).
Azure Search is a search as a service solution. You populate the service with your data, and then you add search capabilities to your web or mobile applications that call the service to search that data. Microsoft manages the search infrastructure for you and offers a 99.9 percent SLA. You can scale to handle more document storage, higher query loads, or both.
You can search your data using the simple query syntax that includes logical operators, phrase search operators, suffix operators, precedence operators, and so on. You can also use the Lucene query syntax to enable fuzzy search, proximity search, and regular expressions. Data integrations allow Azure Search to automatically crawl Azure SQL Database, DocumentDB, or Azure Blob storage to create an index for your search.
At this time, 56 languages are supported. Azure Search can analyze the text your customer types in the search text box to intelligently handle language-specific terms such as verb tenses, gender, and more. You can even enable autocomplete for the search text boxes. Additionally, Azure Search includes geo-spatial support so you can process, filter, and display geographic locations. This means you can show search results ordered by proximity, such as the closest Starbucks.
Azure Service Bus is a managed service for building reliable and connected applications (either onpremises or in the cloud) leveraging messaging patterns. Service Bus is often used as a key component in eventually consistent solution architectures—providing asynchronous messaging integrated with additional Azure resources such as SQL Database, Storage, Web Apps for App Service, or applications hosted on Azure Virtual Machines.
Service Bus features four different communication patterns:
Azure Event Hubs is a highly scalable managed service capable of ingesting millions of events per second, enabling you to capture, process, and analyze massive amounts of data originating from connected devices (often IoT scenarios) and applications. You can think of Event Hubs as a gateway, or entry point, for an event processing pipeline. Data is collected into an Event Hub, then transformed and stored. You have control over what data transformations and storage are needed.
The programmatic interface for Event Hubs is AMQP (Advanced Message Queuing Protocol) or HTTP(S), making it very easy for a wide range of clients to publish event data to Event Hubs. To support the need for massive scale, Event Hubs uses a partitioning pattern to scale the load internally. Receiving messages from an Event Hub is handled via consumer groups. Consumer groups are responsible for knowing from which partition to read and maintaining a view (state, position in the stream, etc.) of the Event Hub.
You will often see Azure Event Hubs used to ingest data in a big data or IoT scenario. A characteristic of both scenarios is the generation and processing of large volumes of (often relatively small in size) data. To process and analyze the data, another Azure service, Azure Stream Analytics, is often paired with Event Hubs.
It is important not to confuse Event Hubs with Azure Service Bus queues or topics. While the two are similar in that they are both messaging systems, Event Hubs is designed specifically for handling message events at high scale. It does not implement some of the messaging capabilities of Service Bus queues and topics, such as dead lettering, filters (property based routing), and various message retrieval, delivery, and scale semantics. Service Bus is better suited for per-message needs, while Event Hubs is better suited for event streaming needs.
While Event Hubs allow you to take in millions of events per second, Azure Notification Hubs send data in the other direction—they enable you to send push notifications to mobile devices from any backend, whether in the cloud or on-premises. With a single API call, you can target individual users or entire audience segments of millions of users across all of their devices.
Push notifications are challenging. In general, the app developer still has to do much of the work to implement even common push notification scenarios, like sending notifications to a specific group of customers. To make them work, you have to build infrastructure that is complicated and, in most cases, unrelated to the business logic for the app.
Notification Hubs remove that complexity, eliminating the need for you to manage the challenges of push notifications. Notification Hubs are cross-platform—they can be used to support Windows, iOS, Android, and Windows Phone apps; they reduce the amount of push-specific code you have to put in your backend. They are fully scalable, allowing you to send notifications to millions of devices with a single API call.
All of the functionality of a push infrastructure is implemented in Notification Hubs for you. The devices only have to register their PNS handles, and the backend can send messages to customers without worrying about the platform the customers are using.
Azure Media Services enables you to provide audio or video content that can be consumed ondemand or via live streaming. For example, NBC used Azure Media Services to stream the 2014 Olympics (http://blogs.microsoft.com/blog/2014/02/06/going-for-gold-windows-azure-mediaservices-provide-live-and-on-demand-streaming-of-2014-olympic-winter-games-onnbc/#sm.00001fhr9yr2zfciwlu2fkqhgu8kp).
To use Media Services, you can call the .NET or REST APIs, which allow you to securely upload, store, encode, and package your content. You can build workflows that handle the process from start to finish and even include third-party components as needed. For example, you may use a third-party encoder and do the rest (upload, package, deliver) using Media Services.
Media Services is easy to scale. You can set the number of Streaming Reserved Units and Encoding Reserved Units for your account. Also, although the storage account data limit is 500 TB, if you need more storage, you can add more storage accounts to your Media Services account to increase the amount of available storage to the total of the combined storage accounts. And last but not least, you can use the Azure CDN with Media Services for the fastest content delivery possible.
Azure Backup is a backup as a service offering that provides protection for physical or virtual machines no matter where they reside—on-premises or in the cloud. Azure Backup encompasses several components (Azure Backup agent, System Center Data Protection Manager [DPM], Azure Backup Server, and Azure Backup [VM extension]) that work together to protect a wide range of servers and workloads.
Azure Backup uses a Recovery Services vault for storing the backup data. A vault is backed by Azure Storage (block) blobs, making it a very efficient and economical long-term storage medium. With the vault in place, you can select the machines to back up and define a backup policy (when snapshots are taken and for how long they're stored).
Azure Backup can be used for a wide range of data backup scenarios, such as the following:
Even though Azure Backup and Azure Site Recovery share the same Azure portal experience, they are different services and have different value propositions. Azure Backup is for the backup and restore of data on-premises and in the cloud—it keeps your data safe and recoverable. Azure Site Recovery is about replication of virtual or physical machines—it keeps your workloads available in an outage.
Azure Site Recovery (ASR) provides a disaster recovery as a service solution for Hyper-V, VMware, and physical servers, using either Azure or your secondary datacenter as the recovery site. ASR can be a key part of your organization's business continuity and disaster recovery (BCDR) strategy by orchestrating the replication, failover, and recovery of workloads and applications if the primary location fails.
While there are many attractive technical features to ASR, there are at least two significant business advantages:
The recovery plans you create with ASR can be as simple or as complex as your scenario requires. They can include custom PowerShell scripts, Azure Automation runbooks, or manual intervention steps. You can leverage the recovery plans to replicate workloads to Azure, easily enabling new opportunities for migration, temporary bursts during surge periods, or development and testing of new applications.
Azure Key Vault is used to safeguard cryptographic keys and secrets in hardware security modules (HSMs) and allows Azure applications and services to use them. For example, you might use Key Vault to store storage account keys, data encryption keys, authentication keys, .PFX files, or passwords.
You can use Azure Active Directory (Azure AD) to control access to a Key Vault, which means you can control access to your keys and secrets using Azure AD. We talked about one example in Chapter 4, "Azure Storage," where you can store your storage account keys that are used by a service principal (an identity representing an application) into an Azure Key Vault and give access only to that service principal, thus protecting your storage account keys.
You can generate keys using Key Vault, but you can also store keys you have generated outside Azure. For security purposes, Microsoft cannot see or extract your keys. There is also logging capability that allows you to monitor the use of your keys in Key Vault.
The list of Azure services in the preceding pages is a sampling of the many services available in the Azure platform. Azure moves at a rapid pace, and new services and features are offered frequently. The rapid pace of innovation is one of the many fun aspects of working with a dynamic platform like Azure.
You are encouraged to review the main Azure site at http://azure.microsoft.com to learn more about the many services available.
Also, there is a web application that shows the many services of Azure and allows you to drill down to learn more.