Azure for developers

Virtual Machines (IaaS)

If you want a bare server with only an operating system or if you already have a virtual machine image you want to host inside the cloud, Azure virtual machines are what you need.

If you don’t already have an image and you want a fresh system, you can choose from various versions and editions of Windows Server, Ubuntu, CentOS, CoreOS, OpenSUSE, Oracle Linux, SUSE Linux Enterprise, and more. More options are being added every month, in fact. If you want a virtual machine prepackaged with licensed software you can use right out of the box, Azure offers a gallery of preconfigured systems (see Figure 1-1). There are dozens of existing images you can choose from and have running in minutes. The choices here are very broad: they include database servers like SQL Server, Oracle, Hadoop, Couchbase, and others; NAS and security devices; application servers like IBM WebSphere and Jenkins; and SysOps tools like Puppet and Docker. More are being added all the time, and it is likely that if you want a specific type of server on a virtual machine, Azure already has an image ready for you to use. It is as simple as click and go.

Maybe you already have a production application that you want to move off of your own data center, but you don’t want to rearchitect your app for a different type of hosting. In this model, you can take the images of your existing Windows and Linux virtual machines and upload them into Microsoft Azure as they are. To do this, you need to create disk space for them in Azure storage and upload the virtual machine images using PowerShell; then you can find your images in the gallery, where you can create instances of them with the hardware configurations you like. This is an advanced process, and you can find full instructions in the Azure documentation.

Virtual machine disk drives are basically virtual hard drives stored as page blobs (more on that later) within Azure storage. Azure also supports network drives that can be mounted to your server instances through a service called Azure Files. Azure Files drives can be accessed by more than one machine at a time and are billed at a separate rate. Either way, you should be able to configure your virtual machines as you need them with some combination of these drive types.

When you create an instance of a virtual machine, you have to choose the tier of hardware on which to deploy it. The price breaks down based on the specifications of each hardware configuration. There are two general categories, Basic and Standard. The Basic virtual machines are designed to handle basic workloads. They do not support load balancing, autoscaling, or any high memory configurations. These machines are available in the configurations listed in Table 1-1.

The Standard virtual machines have higher-rated hardware options and allow for autoscaling and load balancing. They have configuration options in three series: A (standard virtual machine configurations), D (high-performance configurations), and G (the highest-performance configurations available). The standard A series configurations are listed in Table 1-2. The A8 and A9 options come with specialized 40 Gbit/s network connections for extremely high network throughput. The A10 and A11 options are specialized for compute-intensive workloads and come with Intel Xeon E5 processors. 

D series virtual machines run on higher-performance hardware to support intensive workloads and high-performance software systems. They feature a 60% faster CPU, solid-state drives (SSDs) for local storage, and high memory configurations (see Table 1-3).

G series virtual machines (Table 1-4) are the newest offering from Microsoft and are built on the highest-performing hardware available. These machines feature Intel Xeon E5 v3 processors at up to 32 cores, twice as much memory, and about four times as much local SSD storage than the D series machines.

Cloud Services (PaaS)

If you are architecting a new application for the cloud instead of using existing virtual machine images, then Cloud Services is a good option for you. This cloud hosting model maximizes your scalability by automatically deploying your application onto a server and managing its life cycle (for example, by auto-recycling the server if your application fails for some reason), but also giving you a lot of access to the underlying virtual machine. Cloud service servers are accessible through the Remote Desktop Protocol (RDP). They support applications written in .NET, Java, PHP, Python, Node.js, and Ruby. Cloud services can be implemented in two forms: web roles and worker roles.

Web roles are servers that have IIS enabled and are able to start a web application within IIS as part of their startup process. Since this role type is designed specifically to host a web application, it does not start receiving traffic from the Azure load balancer until the web app in IIS is actually running too. Likewise, if the application in IIS crashes for some reason, the web role will be taken out of service and recycled. For web roles specifically, the difference between Cloud Services and Azure App Service web hosting (more on that later) is that web roles give you remote desktop access and allow you to install additional services onto the virtual machines.

Worker roles are designed to run applications more similar to console apps or Windows services. They are kicked off inside of an infinite while loop in a method called Run(). If this method ever returns—for example, if an exception is thrown—then the cloud service knows to take the instance out of service and recycle it automatically. Worker roles can still communicate across the network, of course, and they can even host web applications, but you will need to configure the hosting yourself since they won’t be auto-deploying your apps to IIS.

Cloud services also come with built-in support for staging deployments onto a temporary DNS name and IP address so that you can verify that your changes are stable before you replace your running instances with new ones. If the staged instances look good, you can swap them with your current production instances with a single click and your users will start hitting your new instances automatically.

Cloud services are billed at rates very similar to virtual machines. All cloud service servers are Windows servers; there is not a Linux option yet. The A series cloud service servers (Table 1-5) are designed for normal workloads, though they do support some high-memory options. Just like with virtual machines, the A8 and A9 tier options are optimized for massive network throughput with the 40 Gbit/s network connection. Likewise, the A10 and A11 options are specialized for compute-intensive workloads and come with Intel Xeon E5 processors. 

Just like D series virtual machines, D series cloud services run on higher-performance hardware to support intensive workloads and high-performance software systems. The servers feature a 60% faster CPU, solid-state drives for local storage, and high memory configurations (see Table 1-6).

App Service (web hosting)

Azure App Service is a new bundled hosting option that includes four specific features: Web Apps, API Apps, Mobile Apps, and Logic Apps. You don’t have to use all four of the pieces—you can even use just one. By bundling them together, though, Microsoft provides the service as a unit, so you don’t need separate billed services; this also makes it easier to combine them together to build a rich application since they share a common framework.

Azure App Service components (such as individual websites, mobile apps, etc.) are grouped into app service plans within the Azure portal. These app service plans can contain a single component on up to as many as your pricing tier will allow. Think of an app service plan as a web application with all of the different URLs and pieces it might be composed of grouped together inside. For example, you might have a website for your main public site, a secured website for administration, a mobile app, and a public API. Since these are all part of the same overall app, it makes sense to group them under a single app service plan and serve them together on the same virtual machines; Azure will then deploy them together and scale them together. The pricing tiers for Azure Web Apps are actually based on app service plans, and all of the components within an app service will share the resources granted by the pricing tier that contains them.

Unlike the IaaS and PaaS options, Azure App Service has a Free service tier. You can run up to 10 apps per global region completely free, except for any costs you might incur through other optional services like storage or databases. The Free tier lacks scaling and other advanced features, but it gives you a completely free way to experiment within Microsoft Azure and to test out your ideas and concepts. You’re even allowed to start a production system in the Free tier and then scale it up once you need more power.

Azure App Service is billed in five tiers offering increasing amounts of dedicated resources and features. The billing structure for Azure websites is more complex than for cloud services or virtual machines, but it basically breaks down as shown in Table 1-7. The Basic, Standard, and Premium tiers provide dedicated virtual machines for your websites, so they are billed not only by the tier but also by the size of the virtual machine. The Shared tier does not provide dedicated machines, so it is billed per site at an hourly rate.

Azure App Service Logic Apps

Logic Apps is another new offering from Microsoft as part of Azure App Service. These apps are very interesting because nothing like this has been available on Microsoft Azure before. A logic app is a graphically configured workflow process that pipes data between API apps (and other special connectors available in the marketplace, some of which can be seen in Figure 1-2) based on triggers and timers.

Essentially, this type of app lets you compose an integration workflow between your own API apps, third-party web services, and enterprise systems in a web-based editor. I think the real benefit here is integration. Imagine, for example, a trigger that responded to a picture being added to a Dropbox folder that then sent a tweet; or imagine that when a new customer is added into a customer relationship management system an email is generated to a group of employees, a post is added to Yammer, and a welcoming text message is sent to the customer from the business.

File storage

Azure provides three different ways to store files: Blob Storage, Azure Files, and a content delivery network (CDN). Blob Storage is part of the standard Azure storage account service; Azure Files and CDN are separate services.

Message-based storage

Message-based storage systems, like Azure Queue Storage and Azure Service Bus, are designed to enable different tiers of an n-tier application to communicate in an occasionally connected, or asynchronous, fashion. This architectural pattern brings big advantages in terms of scalability: each tier of an n-tier application can operate at its own maximum speed, since it is not waiting on any other tier to synchronously respond to a request. Each tier communicates with the message queue as a broker and either pushes or pulls messages at whatever pace it can handle. Another advantage of this architecture is that not every tier has to be online at the same time for the application to operate. If one or more tiers become unavailable for some reason, either planned or not, the other tiers can still communicate with the message queue without interruption, and the system will recover when all the tiers come back online.

Service bus

The Azure Service Bus is also a type of message queue, but it brings some more specialized features along with it. Specifically, the Azure Service Bus can be used as a queue, as a message relay, and as a publish/subscribe message system with topics. Service Bus accounts can be created in one of two performance tiers, Basic and Standard. Their features break down as illustrated in Table 1-10, you can find more details on the Azure website.

Table 1-10. Service Bus performance tiers

Feature	Basic tier	Standard tier
Queues	Yes	Yes
Scheduled messages	Yes	Yes
Transactions	No	Yes
De-duplication	No	Yes
Sessions	No	Yes
ForwardTo/SendVia	No	Yes
Topics	No	Yes
Relays	No	Yes
Event Hubs	Yes	Yes
Brokered connections included	100	1,000

A brokered connection in this context is the type of connection where the Service Bus is storing messages (i.e., brokering them), so that not every endpoint in the communication path has to be connected at the same time. When a client connects to the service, it can receive messages that were sent while it was offline. Azure bills these connections based on the number of concurrent connections at peak time during a month. The Basic tier Service Bus accounts have a hard limit of 100 concurrent connections, and the Standard tier accounts allow 1,000 concurrent connections. Extra connections are billed on a rate schedule that you can see on the Azure portal site, but only Standard accounts can exceed the limit. Service Bus utilization, measured in millions of requests, is also billed on a rate schedule.

In the message relay mode, the Service Bus can relay queue messages across the boundaries of firewalls between the cloud and an on-premise application (see Figure 1-3). It does this without punching holes in the firewall, since the relay must be opened and initiated from behind the firewall. This is the canonical example of a hybrid application, partly in the cloud and partly on-premise. By utilizing a Service Bus relay, you can create a queue-based communication pathway between a local app in your data center, even a legacy app, and other services in the cloud. You could use a solution like this to phase your application into the cloud in stages instead of all at once.

In the topics mode, you can use the Service Bus to create topics to which consumer services can subscribe to receive messages. Each time you push a message into a Service Bus topic, every subscriber receives the message as a broadcast (see Figure 1-4). This makes the communication one-to-many, as opposed to one-to-one.

NoSQL data storage

NoSQL databases, also known as Not-Only-SQL databases or NewSQL databases, are rapidly rising in popularity. Instead of featuring data normalization, strict transactional consistency with record locks, and vertical scaling through more powerful hardware, NoSQL databases embrace nonrelational data with loose schemas, multiple versions of records to avoid locking on updates, and horizontal scaling through distributed server nodes. This is a paradigm shift for most developers, since the traditional application development model starts with a relational database and builds everything on top of that foundation. In today’s world, however, where web-scale applications require performance beyond what a traditional relational database can offer (even on high-end hardware), these new data storage solutions have risen to meet the challenge.

Performance is not the only reason to use one of these databases, though; they also bring exciting features to developers to boost productivity and make development more enjoyable. These databases are designed to run on schema-free data models so they can store and retrieve data of any shape, and they are nonrelational so retrieving data does not often require joins or subqueries. This makes application changes much easier, since no migration scripts are required to change tables, keys, or columns and rigid object-relational mapping (ORM) tools are no longer necessary for representing data entities as objects. As you update your application, you can easily store and query entities with a new shape right alongside older entities that look different.

Microsoft Azure offers two specialized NoSQL database options for creating massively scalable applications: Table Storage and DocumentDB. Table Storage is part of the basic storage service, and DocumentDB is a separate service. They each scale and perform differently and are designed to solve different types of problems. There are also many other NoSQL database options available through the Azure marketplace, including a Microsoft-managed Hadoop service called HDInsight.

SELECT * FROM Person WHERE Id = 42;

SELECT * FROM Person WHERE Id < 4000;

SELECT AVERAGE(Temperature) FROM Readings;

SELECT Temperature, Pressure, FlowRate FROM Readings WHERE Id < 4000;

SQL database

Even though NoSQL database systems are exciting and new, relational database systems are still important and relevant for most applications. Azure SQL Database is a managed database service that supports the majority of SQL Server functionality, including database programmability, SQL Server Management Studio, and the familiar T-SQL operations you are used to. Unlike SQL Server on a virtual machine, however, Azure SQL Database supports elastic scale, geo-replication, and point-in-time restore; it is automatically replicated, security patched, and has a guaranteed service-level agreement (SLA). You can scale your throughput up and down as needed to balance cost and performance.

Azure SQL Database was formerly known as SQL Azure and was initially offered as a more basic SQL service with price based on database size. The original implementation of this service lacked some common features, did not perform as well as many expected, and got a bad reputation in some circles. Things have changed, however, and with the guaranteed throughput units and service tiers now available this service is well worth your consideration and production use. In fact, having all of the database management tasks taken care of as a service makes this an attractive option for nearly any application.

Database throughput units (DTUs) are based on a blended measure of CPU, memory, reads, and writes. These units are a bit hard to describe, but they are accurately relative to one another: 10 units will offer exactly twice the performance of 5 units. See the following two tables for more details on real-world performance.

The SQL Database performance tiers are each designed around certain levels of precise, predictable throughput. The Standard tier is designed for a predictable amount of performance in transactions per minute. The Premium tier is designed for a predictable amount of performance in transactions per second. This level of guaranteed performance is impressive and allows you to develop your application knowing you can count on the level of performance that you need.

Wrapping up

Enough with the background. You should now be familiar with all of the pieces Azure provides for creating and hosting your own app, but that’s only part of the picture. What you need to know is which pieces to choose when you’re starting a new project. The answer, of course, depends on what sort of application you want to build. I’ll go over a few common situations so you can extrapolate the best features for your project, starting with a few simple web applications.

Requirement 1: High utilization

One reliable way to guarantee performance under high load is to separate your app into disconnected tiers that can each operate at different performance levels. This ensures that a fast and responsive web frontend is never directly waiting on a slower database–transaction–driven backend while it is serving responses to web browsers. The web tier is allowed to run fast while the database tier is allowed to run a little slower. The way you connect these tiers together is through a brokered messaging system (Figure 1-5). Both ends of this architecture, front and back, can scale independently of one another to avoid performance bottlenecks. The message broker is the key to this pattern. The frontend can send messages to the broker asynchronously, as fast as needed. The backend pulls messages from the broker as fast as it can, but its speed does not have to match that of the frontend since neither tier is waiting on the other to proceed.

To achieve this architecture in Microsoft Azure, you will need a scalable web frontend, a message broker, and a scalable backend. There are a lot of different ways to accomplish this, but there are specific options that will help you to meet your requirements better than others.

Requirement 2: Wide variety of products

There is an architectural pattern known as polyglot persistence, which is another way of saying that an application uses multiple different database types to store its data. For our retail sales web application, we can use this polyglot persistence pattern to store different types of data in specific database types that can best handle the nuances of that data.

Since our retail application stores many different types of products, storing this data in a database that requires a uniform schema across all products is somewhat problematic. A document database is ideal to help solve this problem since it stores documents, normally formatted as JSON, that can exist in any shape so long as they have a key field. If you want to store a book product, you can store it in its own natural shape; give it a page count, authors, editors, illustrators, and so on. Then you can store another product, like a laptop computer, with completely different properties, such as battery life, screen size, processor specs, and memory. Not only can the properties be unique for each document, but the properties can naturally be objects on their own. For the laptop example, the processor property could be an object with its own properties like clock speed, L2 cache, and manufacturer. Documents in this type of database exist as hierarchical structures that can contain nested objects of nearly any shape. A database like this is not suited for complicated relationships, though, so instead of joining tables (or documents) together you will want to store all of the data that a document needs together inside that document. For example, instead of storing a customer in one document and that customer’s addresses in another document, you would store the customer with an object array called addresses that contained all of the address data inside.

For our example, it still makes good sense to store product inventory and orders in a relational database that can enforce transactions and maintain a consistent state. Product information, however, conforms much better to a document database, where each product’s unique properties can be stored appropriately without affecting other products.

If you already have an investment in a different document database, such as MongoDB or RavenDB, you can still use that within Microsoft Azure. Both of these, and more, are offered through external vendors in the Azure marketplace. I recommend DocumentDB over these others because it is offered as a fully managed and scalable service that can be purchased in small incremental units, it supports a SQL-like syntax, and it guarantees a specific performance level based on purchased units that can scale alongside your application.

Requirement 3: Fast search

As a developer, you know how much users depend on search engines and search functions within your own apps. Providing search functionality can take many forms. You may like to use a database option like full-text indexes with SQL queries, or you may use an indexing service like Apache Solr. Either way, I think you would agree that creating your own search service is hard work and takes up time that can be better be spent elsewhere.

For a web application like our retailer site, we want a search service that can scale without a lot of manual intervention on our part. I certainly recommend using an indexing service, as opposed to creating your own database-built solution. There are a few externally hosted options in the Azure marketplace, including Apache Solr, but there is also a new offering from Microsoft called Azure Search.

Azure Search is very new, so I suggest you weigh your options carefully, but the fact that Azure Search is a managed service that can scale elastically without the need to purchase expensive blocks of dedicated server space makes it an appealing option. Also, although it is new, it comes equipped with full-text search, weighted results, autocomplete suggestions, geospatial data, and faceted navigation. This is a rich feature set, to be sure.

Requirement 4: Product recommendations

To drive product recommendations, your best tool will be a type of NoSQL database known as a graph database. This type of database is designed to store nodes of data that are connected to one another through relationships. A database like this is optimized to traverse these relationships to quickly solve queries such as “What is the shortest route between nodes A and B?” or “Which nodes are connected in two or fewer hops that are also connected to nodes C and D?” This type of analysis is ideal for locating products that some customers have purchased alongside products that a current user may have in the shopping cart.

To make this work inside Microsoft Azure, your options are more limited than with the other components we have discussed so far. Of all the graph databases in common use today, only Neo4j is available from the Azure marketplace, and it is only offered as a virtual machine image. There is nothing wrong with this, of course, but it does mean that you will be responsible for managing the operating system and database configuration yourself since it is not a managed service.

Online retail store complete architecture

Putting this all together, you can use the Azure components we have discussed here to create an online retail store web application that can scale to meet customer demand, even under intense load; that can offer all of the advanced features the business requires; and that minimizes the requirements on you as the developer. The diagram in Figure 1-6 puts all of the component pieces together.

Note how the application feeds all data writes into Queue Storage, where the worker roles can handle keeping all of the data stores synchronized. Data reads flow in the other direction, from the databases up to the web roles, and do not involve the backend worker roles. If you prefer to wrap your data access in a service layer, I recommend creating a different set of worker roles dedicated to that function.

One thing I didn’t mention before was how to store the shopping cart data for each customer as they shop around the site. This is something you may have stored in session state in the past, or maybe in a relational database, but neither of those options fits the scalable cloud model properly. One very good option here is to use a key/value NoSQL database. This type of database is designed to simply load data based on a set of keys; no firm schema is required, and it can naturally do this very fast and on a large scale. Two good options here, for storing session state, are the Redis Cache service or Azure Table Storage. Of these two, Table Storage is more cost effective and simpler to use and can perform very well for this type of direct lookup. Once a shopping cart is emptied through checkout, you can simply delete it from the database. This would be accessed and managed directly by the frontend tier; the backend worker roles would not have anything to do with this.

Hosting options

With this type of application, a cloud service is unnecessary (especially at first) since you do not need any access to the operating system and you don’t need to install any custom software on the underlying virtual machine host. In this case, you can use Azure App Service with Web Apps. It is a friendlier, more feature-rich and lower-cost hosting platform that is optimized for websites and HTTP APIs when all you need is an FTP endpoint for management and deployment and you want to deploy through source control. This hosting option also makes it very easy to add new features to your site, such as an API or mobile app, all under the same account and service tier.

Don’t forget that one of the biggest benefits of the cloud is being able to right-size your consumption. Only use what you need, and scale up or down to match the cost to your usage requirements. Table 1-15 will help you decide when to pick Cloud Services or App Service Web Apps based on the features you need.

Azure App Service is available in a wide range of service tiers. If you do not need a guaranteed SLA, you can use one of the multitenant options in which your site (or sites) run alongside other sites from other users on the same virtual machines. This multitenant hosting is available in two modes: Free and Shared. The Free mode has no cost and offers no special features or scaling. The Shared mode allows for custom DNS names and a limited amount of scale. If you want dedicated servers that are only running your sites (not multitenant), you can choose from three levels of dedicated options, Basic, Standard, and Premium, offering progressively better performance and scale.

You should note that Azure App Service, even at the highest dedicated performance tier, only allows for a scale of up to 50 instances (unless you call Microsoft for an exception). This should be sufficient to handle a very large amount of throughput and number of concurrent users, but remember that you can also move to Cloud Service web roles. Since web roles are preconfigured as web servers and support all of the same programming languages, this move will be very easy. You should consider this the natural next step if your websites ever do need to cross that instance limit or if you need direct access to the underlying operating system.

Requirement 1: Web API

Since we are using the Azure App Service, you might naturally suspect that an API app would be the right choice as a backend to our SPA site. Unfortunately, it is a bit more complicated than that, for a few reasons. First of all, our SPA site is hosted from a web app with a certain Internet address. An API app, even in the same hosting plan as the web app, is going to have a slightly different Internet address. Web browsers are designed to enforce a security measure called the “same origin policy,” which means that JavaScript in the web page is not allowed to talk to a different server than the server that sent it the original web page. This means that the JavaScript frontend will not be able to talk to the API app without you setting up security workarounds, which you should normally avoid. Secondly, unless you are using OAuth for authentication, you may have to authenticate to both the web app and the API app separately, which would be another headache.

Both of those problems could be resolved through some advanced configuration, but why fight with that when there is an easier way? Rather than using a separate API app for your backend, what you need to do is to create a web API within your web app. This is the normal solution anyway; I just wanted to make sure you understood why API apps are not a good option for this scenario. For example, if you are using ASP.NET MVC to write your web app, adding some API controllers is as easy as a right-click in Visual Studio.

Requirement 2: Efficiently serve static content

Since this app will require scripts and other static content, such as stylesheets and images, to be highly available and quickly downloaded, it would be a good idea to host these in a location other than the same web server as the site. This way, the client web browser can download this content concurrently and cache it for repeated use and access. One option here is to use Blob Storage to host this content over HTTP. A better option, however, would be to use a content delivery network service so that users always access this content from a server geographically local to them, for the best possible performance. This is especially important if your site is serving customers globally, but it still helps for serving users across the same continent. Microsoft Azure does provide a CDN service that would be a good fit for this solution.

Requirement 3: JSON data layer

For this solution, you can design your data layer in whatever way makes the most sense. If you use a traditional database model, you will need to use a web API layer to translate between the database and the JavaScript frontend. This is a normal pattern, but it comes with some performance trade-offs and possible bottlenecks. Since the frontend of this application is built on JavaScript, you can avoid that problem by using a JSON-based document database, like Azure DocumentDB, instead. Specifically, since the database and the frontend both utilize JSON for manipulating data, you can pass data directly from the database up to the frontend without any translation or mapping. In effect, you can simultaneously simplify and speed up your application at the same time.

You will also probably need a Table Storage account for handling features like session state (if your app needs it) or other key/value-optimized data, like a shopping cart or user configuration settings. Table Storage can be used to store any type of data, so you can store the appropriate JSON documents there, and your frontend will be able to consume them easily.

Your architecture will look something like Figure 1-7.

Hosting options

Since we have already covered the web hosting options of Cloud Services and App Service Web Apps, we can just discuss which of them is more appropriate here. Honestly, you could go either way, but I would suggest starting with Azure App Service since it offers more productivity features, has a built-in framework for adding mobile apps, and can easily be migrated to Cloud Services if you ever need to. The only reason I would choose Cloud Services initially is if you need to install additional software on the servers, such as a Windows service, or if you really need remote desktop access. On a similar note, if you need a Linux server or if you need to install some third-party software that cannot be loaded through an .msi file, you will need to host your site on Azure Virtual Machines.

Requirement 1: Write-intensive workload

In order to support a write-intensive workload that can also handle fast reads, you need a database that doesn’t lock records and that can scale horizontally into multiple nodes. A relational database can sometimes be configured with looser rules for record locks, but it is not the ideal choice for horizontal scaling. You’re better off in this case choosing a distributed NoSQL database. Here are some of the criteria to consider in deciding which database to use:

• You will normally be querying for all of the posts for a single user rather than joining data from multiple users at the same time.
• Each user will potentially generate hundreds or thousands of posts and the list will continue to grow over time, so you need a database that can easily retrieve all of these posts for a user very efficiently.
• It is all right if there is a short delay between the time when one user posts a message and when other users see it.

Based on these criteria, I think the ideal database is a column-family database that organizes users as rows and user posts as columns so they can be retrieved together at peak efficiency. This database type supports horizontal scaling and eventual consistency, so it can support a write-intensive workload while still serving fast reads.

To implement a column-family database in Microsoft Azure, your best option is to create some Azure virtual machine instances and install the database software yourself. A great choice here is HBase, as discussed in Hadoop and column-family databases.

I should mention that using a column-family database requires a lot of manual maintenance and customization. You may prefer to use a database like DocumentDB to store your data instead. DocumentDB is a very high-speed and scalable service that can work for this application’s type of data so long as you carefully design your data entities and queries. Whether you implement a column-family database or a document database, you will need to work out your data querying strategy carefully to get the correct level of performance.

Requirement 2: Facilitate user relationships

As discussed in our first example, the online store, the best way to implement tracking relationships is with a graph database like Neo4j. This type of database is optimized for traversing connections between data nodes—in this case, users of your social media site and their relationships such as “friend of,” “classmate of,” “employer of,” and so on.

To implement a Neo4j database in Microsoft Azure, you can use the virtual machine gallery image for Neo4j and get started right away. Since this is a virtual machine image, you will be responsible for managing and configuring it yourself, but the image is provided for you to get you started.

Requirement 3: User search feature

In this app, users will primarily be searching for one another by name or by some other key value, like a phone number or email address. There isn’t a need to weight or filter search results by an advanced set of criteria. To implement this search functionality, I recommend using Azure DocumentDB as both the database to store user information in general and the search engine to find users. DocumentDB will automatically index all the properties in the user records and will enable fast search by all of these fields.

Requirement 4: Big data analysis

As your app becomes more popular, you will probably want to use the data that your users are generating for the purposes of your business. With the permission of your users, of course, you could start to roll out some advertising space based on data or trends in their posts. Or you could process anonymous data in messages and posts to determine popular and trending topics to enable features like breaking news or important stories. Either way, you will need some tools to implement this functionality.

Hadoop has become the industry standard for processing big data, and Microsoft Azure provides a full-featured Hadoop service called HDInsight that should fit this requirement well. One great benefit of this service is that it is managed within Azure as a scalable service, so you do not have to set up the virtual machines or databases yourself. You just access the service like you would DocumentDB, Table Storage, or SQL Database, and you don’t need to worry about all of the backend maintenance.

Social media application complete architecture

This application requires a good design incorporating polyglot persistence, something like the diagram in Figure 1-8.

Requirement 1: Support Android, iOS, and Windows

To start a mobile app for Microsoft Azure, you can log on to the Azure portal and create an App Service mobile app instance that can host the backend of your system. Once you have done this, you can download templates for Windows, iOS, and Android. Figure 1-9 shows the templates available at the time of this writing; the Xamarin options are C# projects that compile to native code. These project templates include both the server-side project in the language you chose and the mobile app project for the platform you chose. The portal also gives instructions on how to connect existing apps in these platforms to your new mobile app backend.

Currently the templates do not include Apache Cordova for Android apps, but I suspect that this will be added soon, along with other templates.

Requirement 2: Support Push Notifications on All Platforms

Azure App Service mobile apps automatically include push notification support for Windows, iOS, and Android (Figure 1-10). These push notification services can be configured from the Azure portal by adding the configuration information you will have received from the different app stores.

If you have your own mobile application backend tier and you want to plug in just push notifications, you can do that too. If you want to leverage a service that sends push notifications, to all of the users of your mobile apps, you can utilize the Azure Notification Hubs service to enable and manage that. This service provides connection options so you can send notifications to any number of your mobile users, no matter what platform they are on (such as if you have deployed your app across multiple app stores). Notification Hubs comes enabled for push notifications to Windows devices, Apple devices, and Google Android devices, all of which can be enabled and configured through the Azure portal.

Requirement 3: Support Offline Sync on All Platforms

Offline sync, just like push notifications, is implemented differently across the different mobile platforms. Fortunately, Azure App Service has already implemented this in the templates for each mobile platform you can download from the Azure portal.

Mobile Engagement

Microsoft Azure also has a new service called Mobile Engagement that provides you with an API you can integrate into your mobile apps to capture some specific big data analytics and to provide some targeted push notification and communication options. This set of tools is designed to increase the return on your investment by helping you get to know your users and their habits better so you can better market services to them, engage with them, and retain them over time, which all lead to generating more revenue.