The Rise Of Clouds

This blog post introduces cloud computing, the trends and business benefits that are driving it and also pointers to future blog posts on the “state-of-the-art” cloud computing .

Cloud Computing:

Clouds are one of the three C’s – Cores, Coders and Clouds – that may very well mark the next crisis or opportunity for our IT ecosystem (Remember that crisis in Chinese = danger + opportunity). As such, they are getting lots of attention and social discussions in the popular press these days.

What are Clouds?

Clouds – Cirrus, stratus, altostratus, cumulus - they are the scientific names of the common clouds. They drift across the sky, reflecting the changing wind and weather. Similarly, “clouds” in computational science world advance, representing a game changing scenario for all IT ecosystem stakeholders.

The concept fulfills the long-held dream of computing as a utility and thus represents an inflection point in the geography of computation and IT service delivery. It tries to replace the traditional ‘desktop-as-a-platform” with “internet-as-a-platform”, which can dynamically provide always on and always available IT services to address a wide range of needs, ranging from scientific research to e-commerce, over the web.

The key idea here is to provide a utility service, similar to a power grid, into which a user may plugin regardless of location to access centrally located services. One other related utility services such as is this, is banks. Banks let customers to visit any ATM and access money from their bank accounts wherever they are. Similarly, clouds provide computing resources as if it were a utility such as electricity – accessible by anyone, anywhere with a network connection in an on-demand, highly available, scalable and “pay-per-use” basis.

To achieve these goals – infinite scalability, guaranteed performance and nearly “always-on” availability demands – cloud platforms build on decades of research in virtualization, parallel and distributed systems, utility computing, networking, web and software services.

Driving Trends

Like most technologies, this paradigm also evolved from a need. Some of the trends that made clouds possible include:

(1)Growth in the diversity of sources, data rates, and the amount of transient and archived data.
The tremendous amount of information of value available in electronic form – via digital sensors, communications, computation, and storage mechanisms- brings previously unknown data processing and analysis challenges. Figure 1 shows a sample of the “Big Data” problems that we are talking about.

cloud and big data

cloud and big data

(2)The increasing power, decreasing costs of commodity hardware,and availability of ubiquitous broadband.
The near exponential gains in performance and plummeting costs of commodity components such as the rise of multi-cores, the growth in storage density and decreasing per-GB costs make it possible to build low?cost high?capacity performance systems. Availability of cheap broadband internet access also
contributed to the rise of clouds.

(3)Virtualization, Open source software and the availability of better programming models.
Virtualization brings agility and flexibility to IT infrastructure. Open source software powers cloud architectures through the software stack it contributes including communication fabrics and middle-ware tools. In addition, the emergence of better parallel and distributed programming abstractions such as Map Reduce, Big Table, Google File System helped to achieve reliability, scalability and availability for commodity PCs.

(4)The increasing use of high performance computing for analysis and simulation of big-data in academia and industry.
Industries are increasing their use of high performance computing to reduce costs, decrease time-to-market for new products, and to increase product output and quality. We are also increasingly seeing the impacts of high performance computing in the domains like financial engineering, retailing, drug discovery, film production, grid computing.

Thus, the convergence of need and solution has produced the current concept of cloud computing, which promises to benefit all parties involved.

Some Examples:

  • Infrastructure: Compute facilities provide computational services so that users can use central processing unit (CPU) cycles without buying computers (EC2, gogrid, masso). Storage services provide a way to store data and documents without having to continually grow farms of storage networks and servers (S3,simpledb).
  • Enterprise Apps: SaaS companies offer CRM services through their multi-tenant shared facilities so clients can manage their customers without buying software (Salesforce). Similarly, ERP, Financials, HCM, PLM and SCM.
  • Office Applications:Google Apps, Zoho officesuite
  • Processing pipelines, document processing, image processing, video encoding, indexing, data-mining; batch processing systems, back office applications, log analysis, nightly builds, automated unit testing and deployment testing; websites – instant websites, promotion websites, seasonal websites

Business Benefits

As we see from the nature of examples using cloud architectures,there are clear business benefits including, among many others:

1.Almost Zero upfront infrastructure investment: Utility computing style with no fixed or start up costs and savings in terms of power, space, cooling, hardware and operations personnel.

2.Just-in-time Infrastructure: Infrastructure gets provisioned, re provisioned, deployed and relinquished according to business needs

3.More efficient resource utilization: Effective and efficient management through on?demand resource provisioning.

4.Usage-based costing: Pay-per-use model based billing for the fraction of storage, server capacity or time duration used.

5.Potential for shrinking the processing time: Highly available and elastic. Exploits effective parallelization reducing the total processing time.

Future Blog Posts:

We continue to explore this nature and potential of cloud computing in our blog series. In the course of our blog series, I’ll cover

·Big-data problems and how to tackle them with upcoming and useful abstractions (programming models, software approaches)

·The unreasonable effectiveness of cloud-based data approaches in solving real IT challenges

·Technological implications of cloud-based approaches

·Use of clouds with other disciplines such as machine learning, collective intelligence, e-science and semantic web

Conclusions: Hype and Reality -A Caution

For the sake of “greater good” and saving endless argument battles, we need to separate the hype from the reality. Clouds won’t magically do things- despite all the hype, they don’t fix/solve all your IT problems out-of-the-box. Nor does it solve the “world hunger” problem. Like all new technologies, they challenge some of our computing wisdom, change our assumptions and brings about a paradigm shift in how computing is provisioned, deployed, and metered.