WAN Governance

SYLDAVIA is a great country - but where is the delay?

Sometimes, marketing buzz is really good at creating confusion in people's mind - One may even wonder if it is not its main goal...

I was recently invited to meet the IT infrastructure team of a semi-public organization in a central European country. Let's call it Syldavia (http://en.wikipedia.org/wiki/Syldavia). It is a two or three hundred miles wide country with a few million inhabitants. It is a well organized country, with handsome, friendly and highly educated people. The organization that invited us is in contact with the public. In a nutshell, it runs a network of two datacenters and a few hundreds of branch offices located in Klow (Syldavia's capital) as well as in cities and villages all across the country. The purpose of the meeting was to discuss the various solutions to provide good and sustainable performance to their business applications running at branch offices and hosted in the datacenter.

The first question that raised was: "how could we solve our delay problem?". I must recognize I remained puzzled for a while: Syldavia is a really great country, but certainly not a large one. The maximum distance between datacenters and offices is about 300 kilometers: 1 millisecond at speed of light. Okay, networks are much slower than light (approximately half speed), and we need to add access modems (SDSL in this case); but I doubted the total delay could be much larger than 10ms.

Fortunately, I remembered that Bordurian spies (http://en.wikipedia.org/wiki/Borduria) made a recent attempt to brainwash our Sylvanian friends, using the old good FUD (fear, uncertainty, doubt) tactic: delay here, delay there, everything is about delay, solve this delay problem or you are a stupid manager, etc...

A more objective discussion demonstrated easily that delays, if any, would be caused not by distance but by congestion: saturated device buffers would easily store a few seconds of traffic, leading to poor user experience at branches. Congestion avoidance technologies (like traffic priorization, redundancy elimination or bandwidth upgrades) are then better suited than brute-force acceleration boxes.

To solve this situation, it is clear that a few well-located devices (in data centers and possibly some main offices) would guarantee good and stable performance for all business applications at a much lower cost than deploying Bordurian devices at every branches, with all its associated complexity and cost: it is then possible to satisfy both users and CFO at the same time.

Could I urge you to resist to Bordurian's intelligence services?

Delays are not the root cause of performance, they are symptoms. Real root causes of poor user experience over networks are:

• Geography: distance of long haul networks --> Delays (in the core network) --> Impact chatty applications only --> Protocol acceleration
• Lack of bandwidth (or excess of traffic) --> Congestions --> Delays (in network devices) --> Impact all applications --> QoS, compression, bandwidth upgrades

Viva Syldavia!

All your ducks in a row?

What about twelve questions as a last check in before the final countdown?

 A) Key applications and end-users Quality of Experience (QoE) are likely to be the first topics to think about. You’ll probably need to clarify clarifying the following areas:

1.    What are the key applications for the enterprise? This is usually a question that IT managers can answer easily: ERP (SAP, Oracle…), voice and tele-presence, collaborative tools (like Sharepoint or Notes…) and a few others. If the list is too long (say more than 10), there is a flaw: as everything is critical, there is a fair chance that none are truly important…

2.    What are the different application delivery models? For example: is desktop virtualization (Citrix, VMware…) rolled out? To which extent are the data centers consolidated? Delivery models may differ for each application; they are important as they impact operations and responsibilities in the enterprise.

3.    What does ‘performance’ mean for the business? While both are fundamental for the business, a transactional application that’s intensively used by employees probably has different requirements from email and corporate collaborative tools.

B) The operations model is very much driven by the enterprise’s culture, organization and evolution strategy. It is obviously related to a WAN Governance implementation:

4.    Is IT and network management centralized or decentralized? Is the corporate team empowered to make global decisions, or does the enterprise work in a more collective consensus-driven mode with the Business Units and territories? What is today’s situation and is the enterprise willing to change it?

5.    Is the company culture about outsourcing or insourcing? Some enterprises will decide to subcontract all non core-business activities as much as possible; others will prefer to keep in-house operations.

6.    What is the preferred run model? Self-management (where all the work and responsibilities associated with WAN Governance remain within the enterprise), co-management (technical work is performed by a service provider but control is kept at the enterprise level) or even full management by a third party?

C) Money talks: financial aspects – surprise surprise – have a strong impact on IT and Network managers’ strategy for application delivery over the WAN:

7.    Capex versus Opex. Depending on the company financial structure and policy, the CFO’s preference to minimize Capex or keep Opex as low as possible has to be considered.

8.    What is the desired economic impact? Is the company’s primary requirement to reduce IT and network costs(Yin-face of the economic impact) or is it first looking to improve productivity (Yang-face)?

9.    How are expenses allocated? Will costs be broken down between Business Units? Per Application delivered across the network? Per site, region or subsidiary? Will they be credited centrally? The way network expenses are shared is probably the most meaningful clue to how the WAN is managed.

D) Organization of the Governance is the next important issue to be decided. While it will be awkward to set-up complex processes, some aspects must be clearly defined in order to obtain the blessing of the company management and, finally, the sustainable benefits that are looking for:

10.  Who are the sponsors? It is really useful to set-up a Steering Committee that involves the key stakeholders like the CIO and the CFO, and why not some representative people from Business Units (after all, they are the ones you care about). Frequency or meetings must not be too high – quarterly or even bi-annual meetings and/or reports is probably sufficient.

11.  Who’s in charge? Do not expect the Steering Committee to do the job – it is responsible for setting goals and guidelines and discussing achievements, no more, no less. Thus someone else, probably working in the IT organization, has to be clearly empowered to animate the WAN Governance process.

12.  What are the KPIs? These are not so easy to define, Key Performance Indicators have to be kept as a minimum, simple to understand for non-technical people and relevant to objectives.

While I do would not begin to suggest that answering these 12 questions will guarantee your success, they may be useful guidelines for a first journey into WAN Governance-land. So, ready to go? 3… 2… 1…

Adult, at last?

You may wonder why talking about Autonomic Networking, a quite technical concept, in a blog dedicated to WAN governance and alignment of networks to enterprise business? There are two fundamental reasons for this. The first one is that Autonomic Networking is the key enabler of WAN governance, so we could not ignore it here. The second is that... I like it very much.

Autonomic Networking can be illustrated my many analogies (expect some of them to be posted here time to time). You may read a summary on http://en.wikipedia.org/wiki/Autonomic_networking, and you will easily find many reseach papers if you are interested by details. It is often associated to cousin concepts as semantic networking, traffic-aware networking, self-learning, self-healing, self-management, etc...

The fundamental idea of Autonomic Networking is that it is possible to design distributed systems that automatically collaborate and adapt to fast-changing situations in order to achieve a common goal.

 Looking at enterprise networks, we easily find that, in the one hand:

·         The network (whatever the deployed access and core technology) is a rather static infrastructure. Its main goals are availability and stability. Telcos have defined strict rules about provisioning, traffic engineering, monitoring, etc... This infrastructure is shared among many enterprises that do not care about each other, and just want to share the price of this multi-tenant network infrastructure (which is the 'raison d'être' of Telcos). We can easily understand that a (relative) stability of network parameters is necessary to achieve the expected level of quality. From the enterprise point of view, this means that changes (like adding new rule, tuning a device, increasing bandwidth, adapting CoS settings...) take weeks or months to be applied to the global network.

On the other hand:

·         Enterprise traffic is awfully complex: many applications (up to 100s) x many sites (up to 1,000s) x many users and terminals (up to 100,000s) x business seasonality (monthly financial consolidation, end-of-year closing activity, etc.) generate an astronomical number of situations.

·         Enterprise traffic is very dynamic: networking is about communication between devices that serve applications to people. We are impatient animals, and waiting a few seconds is the most that we can accept. In a branch office, an employee downloads her emails, her colleague starts a transaction, another one begins a phone call, etc... The consequence is that while we can reasonably understand average traffic in the day (stats will work), it is impossible to forecast what will happen in the next second, at any place.

Dynamic and complex demand (traffic) versus static offer (network): here is the challenge that Autonomic Networking has been designed to address. The good news is that nowadays this is feasible: while human brain (when feed with a reasonable amount of caffeine) is a very slow device able to deal with dreadfully complex problems, computers (when feed with proper algorithms) are communicating machines able to solve thousands of simple issues in a single second: associating of the two of them leads to a system that is humanly managed at the objective-level (what result do I want to achieve?) while processors take collectively care in real-time of all the micro-decisions (what should we do right now? And now? And here? And now? etc.) according to actual - and unpredictable - situations in order to meet the objectives.

One of the consequences of Autonomic Networking is the rising of the Objective-based management model, in place of the legacy usage of Policy-based management. This will surely be the topic of another post.

Another consequence is that Autonomic Networking brings network management from its current childhood (devices has to obey) to the maturity of adulthood (devices can decide). 

 

When queues mean queues

As I am often asked to share practical examples of the direct relationship between application performance and enterprise business, I like to share here one of my favorite cases.

This particular business is in the retail market, with many supermarkets full of shoppers. The IT system is used for two very different purposes:

·         Logistics and administration (stock management, pricing, ordering, etc.)

·         Customers’ electronic payment between local terminals and a central application server.

Let's talk about the latter. While it is easy to understand the importance of this ePayment application’s availability for the business, it is even more interesting to drill down into the differences between ‘good’ and ‘bad’. It just so happened that the IT team was smart enough to build a model that directly relates customers’ average payment waiting time to the application response time. This study clearly shows a few thresholds (all have been reached in the past):

·        When application performs well, the waiting time depends mainly on the number of people queuing up. It is managed by varying the number of open payment terminals (opening all of them all the time would cost too much without real benefits, so it is important to match the number of operators to the number of customers present in the supermarket).

·         If the application transaction duration degrades to a certain level, then customers are obliged to wait in line for much longer: their Quality of Experience (QoE) is poor. They feel angry and start complaining loudly: this is not good for the image of the retail chain.

·         When it gets even slower, the most impatient customers start abandoning their shopping carts full of goods. Employees then have to throw out the fresh goods (meat, vegetables…) and replace everything else on the shelves: losses, extra work, less business (and probably lost customers)…

·         The bottom level of poor performance creates such a pressure on the supermarket employees that they are be forced to open the doors and let people carry their stuff away, without paying for it (could be good for the image, but not for the business).

We can see that good and stable application performance is not only a phrase and goal for technical IT and network engineers, but is directly proportional to the enterprise’s business, its image and finally its bottom line. I’m sure you will be able to adapt this story to your own business, and I will be very interested if you would like to share this with us.

The two faces of Economic Impacts

As economic pressure keeps growing, enterprises' CxOs require (almost) systematically studying the financial impact of each new IT project. We propose to discuss this quite complex topic in a suite of posts to come.

Like many (if not all) things in life, economic impacts can be considered under two very different angles:productivity increase (“Yang”)and expenses reduction (“Yin”).

When this question is addressed by enterprises at global and strategic level, the network can actively participate to both of them.

1. YANG: Productivity increase

a) Ensuring an appropriate, predictable and stable level of performance, no matter where or when. The goal is here to improve business operations thanks to the sustainability and predictability of good performance in every circumstance: peak time of the day, end of week activity increase, end of month financial consolidation, antivirus database update and all the many unplanned events that occur on the enterprise network and impact users’ quality of experience. It is possible to achieve this predictability through various techniques like Quality of Service (QoS), application SLAs, network optimization, etc. the mixture of them depending on the nature of the business application (real-time  voice, time sensitive transactional apps, ...) and on the characteristic of the network (its size, reach, access technology, etc.). Objective-based management and Autonomic networking have proven exceptional effectiveness towards such a goal.

b) Maximizing the performance of business applications over the network may reduce users' waiting time and provide them with direct and personal productivity improvements. Often called Wan Optimization, application acceleration is achieved by using different techniques (according to the nature of applications and networks) like redundancy elimination, caching, protocol acceleration, etc.

c) Providing IT agility in order to facilitate strategic transformations of the infrastructure: centralization of application servers, desktop virtualization, collaborative applications, etc. The productivity improvement comes here from a better managed and more cost effective IT organization at the global enterprise level. The network is always a key part and very often the most impacting success (or failure) factor of such transformation, so its contribution must be carefully understood.

2.    YIN: Expenses reduction

a) Bandwidth reduction, would it be immediate decrease or differed network access increase, can be achieved through network efficiency increase and redundancy elimination (compression) approaches. It is current to achieve an x3 reduction factor (or an increase postponed by 3+ years), which drives a significant impact on the network cost.

b) Network operation simplification actively participates to cost reduction, thanks to proactive incident elimination and shorter resolution time. It makes use of active techniques, deep application understanding and crystal-clear visibility and reporting.

c) Investment reduction, the purpose of which being to limit the deployment of dedicated technologies that might have negative consequences in cost, logistics and operation complexity. Asymmetric deployments and different flavor of virtualization are examples of solutions that participate to such reduction by limiting the hardware deployed in branch offices.

We will talk soon about each of these aspects, diving progressively into more details and providing examples and discussions. I expect you to share your experiences and advices in this multi-face area.

10 x 10 = 11

More is more... up to a certain extend. For example, let's have a look to Application Acceleration over the WAN. When you listen to vendors' marketing, it is up to the one that promise the largest possible acceleration (or data reduction) factor: x10, x100, x1000... Who wants more?

Now, back to the reality and consider Mark, an ACME enterprise's employee that works in a branch office, and uses an application which server is hosted in a remote datacenter. Before application acceleration, Mark used to wait 100 sec for the transaction to be completed(I know, this is quite long but it simplifies the discussion).

Deploying vendor Y's technology, Mark gets x10 acceleration factor: now, the waiting time is of 10 seconds, which is far better: the saving is of 90 seconds for each transaction.

Vendor Z comes to Mark's enterprise IT organization and commits to get x100 acceleration factor: 10 times better that Y! What a deal, indeed. After installation, the promise is fulfilled: transaction time goes from 10 seconds down to 1 second: the saving is of 99 seconds per Mark's transaction compared to the initial situation - 10 times better leads to 10% improvement!

I'm not arguing that 100 is not better than 10, but up to which extend, and at which price?

(I let you make the case for vendor W, that is sure to obtain x1000 acceleration factor).

Wan Governance - Day 1...

While IT governance is a relatively well accepted concept (see for example http://en.wikipedia.org/wiki/It_governance), we found no similar notion for the enterprise's network which sounds weird in our current challenging environment. This is the reason of this blog.

We will progressively address the main benefits of Wan Governance for enterprises - which in a nutshell consist of aligning the network to the enterprise's business. We will talk about users’ benefits: guarantee and acceleration of application performance, simplification of network operations, cost reduction, application SLA, KPI...

We will also talk about technology (expect a lot about Autonomic Architecture, customer and provider-edge deployments, etc.) and features (visibility, optimization, acceleration, security...). I expect we will find an appropriate way to demystify some too-well accepted facts.

Oh, I forgot: you will quickly discover that I'm not a native English speaker (nor writer) - so I apologize in advance for the numerous mistakes you will surely find in this blog... Thanks in advance for your patience!