A review of Middleware Vendors Performance & Marketshare from 2004 to 2011

[All content provided on this blog is mine and they do not represent the opinions of my current or previous employers.  The content herein is based on public sources/websites and strictly for informational purposes.]

Over the last decade, middleware vendors’ product and growth strategies were fundamentally changed by JEE and SOA.  Transaction/Database/Integration middleware vendors, both closed-source and open-source, standardized on JEE application servers or Java frameworks (AKA light-weight Java containers) as product foundation.  This helped app-server agnostic vendors with new go-to-market opportunities and gave customers deployment flexibility. Vendors also embraced SOA principles of modular/self-contained services for product componentization, and standards-based implementation for product/vendor interoperability and extensibility.

According to Google Trends, interest in SOA began before 2004. A lot has changed since then.  Different acquisition strategies have consolidated the MW vendor landscape (i.e. IONA, BEA, SUN, Cape Clear, FuseSource, JBoss, webMethods).  The consolidation accelerated by new market trends such as Application Platform-as-a-Service (i.e. SpringSource, Heroku, Makara) and Integration Platform-as-a-Service (i.e. Cast Iron, Boomi).  On the opposite side, some vendors changed product strategy of App+DB+Integration MW to refocus on their core strength (i.e. Progress Software sells off Orbix and Artix to Micro Focus)… Cloud, Big Data, and analytics continue to impact vendors product strategies and service offerings.

In this post, I am going to take a brief look at key MW vendors performance and marketshare based on revenues as reported in their annual reports.  Different vendors have established different periods of their fiscal year.  For example, IBM’s FY 2011 starts form Jan 1, 2011 to Dec 31, 2011.  In Oracle’s case, FY 2012 starts from June 1, 2011 – May 31, 2012.  So, in the following charts, when I list IBM and Oracle revenues for 2011, I am using numbers from IBM’s 2011 annual report and Oracle’s 2012 annual report.  I will update this post with FY2012 / 2013 numbers once annual reports are available.

Vendors Overview
I decided to look at historical numbers for BEA, SUN, Oracle, IBM, Progress, Red Hat, Software AG, and Tibco.  I selected these vendors, because they were/are well established in the enterprise middleware software space and recognized as leaders in the market. 
As stated above, I am using middleware revenues reported by vendors in their annual reports.  In some cases, these numbers include vendors’ revenues on non-JEE transaction middleware (i.e. Oracle Tuxedo, IBM CICS) and / or license or subscription revenues from OS (i.e. IBM zOS, Red Hat Enterprise Linux).  Oracle numbers do not include revenues from Oracle Applications.  They also don’t include vendor’s MW consulting or education services revenues.
Here are the vendors and their reported middleware revenues from 2004 – 2011:
 
In the total row, I have added up annual revenues by all vendors.  I am using this number as a basis to approximate vendor’s marketshare.   There are a couple of observations to make:

1. Since 2005, Red Hat, Software AG, and Oracle have grown middleware revenues by 319%, 169%, and 119% respectively.   During the same period, Tibco increased revenue by 106%, and IBM by 54%.  Progress takes the last place with 31%.   
2. In the case of Red Hat, they have done a fantastic job of maintaining subscription renewals and signing new contracts.  It is important to consider they are a new and different kind of software company, and also they don’t break down the revenues by product areas (i.e. RHEL vs. JBoss) .  However, looking at their numbers prior to 2006 when they acquired JBoss, I am guessing Red Hat’s MW subscription revenue to be less than 20% of Red Hat’s revenue. 
3. Based on revenues above, it is reasonable to assume that IBM and Oracle have a combined MW marketshare of ~ 92%.  Since 2005, Oracle’s marketshare has been incrementally increasing while IBM’s decreasing:

In the next sections, I am going to look at individual vendors performance.  I am excluding BEA and SUN, since they were acquired by Oracle.

Oracle

Oracle has come a long way.  In 1990, Oracle's total revenue was a little over $916M. By 2002, Oracle had already expanded its software portfolio beyond database and database related middleware, with JEE transaction server and Apps.  By then, its software revenues exceeded $7B.

Since 2005, Oracle’s software segment (MW+DB+Apps) has performed much better than its closet competitors with steady YoY positive, annual growth despite global financial troubles and dragging economic growth.  Oracle’s vision and strategy of Fusion and its targeted acquisitions have worked very well.
Looking at Oracle’s current revenue mix, it still remains a software company at its core. Services, which include consulting, education, and on-demand hosting, contribute a smaller percentage to Oracle’s total revenue. With the acquisition of SUN, Oracle entered hardware business. This has enabled Oracle with better execution of its Engineered Systems and powers Oracle Cloud.

IBM

They key business segments in IBM are IBM Global Services (IGS) which includes Global Technology Services (GTS) and Global Business Services (GBS), Systems and Technology Group (STG), Software Group (SWG), and IBM Global Financing (IGF).

Looking at IBM’s revenue mix, IGS and SWG have been key to IBM’s revenue growth over the last 8 years. (Margins on SWG and IGS is about 85% and 30% respectively). More than 50% of IBM’s revenue comes from Services.  SWG makes up > 20%, but  increasing.  N.B. In 2008, for the first time in IBM’s history, SWG exceeded STG in revenue.

Looking at IBM’s revenue by geography, the executive team has done a great job of shifting resources to emerging and developing countries now beyond BRIC.  According to IBM’s 2011 annual report, more than 22% of IBM’s revenue is realized from growth markets.

Here is a breakdown of SWG revenue by product areas:

IBM SWG is made up of different key brands: WebSphere, Lotus, Tivoli, Information Management, and Rational. 
In the chart above, X-Brand MW (Cross-Brand middleware) refers to revenue from all SWG brands.  IBM also sell software for mainframe.  Other-MW refers to mainframe software for zOS (i.e. IMS, CICS).  OS refers to zOS operating systems and AIX. In 2009, IBM sold off its PLM business to Dassault.  And, “Other” refers to SWG lab services.  When comparing IBM’s MW revenue with other vendors, all revenues except for Other and PLM were included.   

In SWG, X-Brand MW has been growing while other segments have been declining.  IBM doesn’t break down revenue by specific products, but attributes growth to specific acquisitions as follows:

• iLog and the Smarter Commerce offerings (Sterling Commerce + Coremetrics + Unica) have been key to WebSphere’s growth.
• In Information Management, FileNet, Netezza, and Cognos have fueled growth in that segment. 
• Telelogic continues to help the struggling Rational brand, and
• Micromuse & MRO have been driving growth for Tivoli.

IBM doesn’t break down revenue by new license vs. renewals, but states that it is generally 2/3 and 1/3 of total revenue for maint renewals and new licenses respectively.

IBM’s SWG growth strategy is centered on Cloud, Mobile apps, Analytics, and generally new product bundles. 
In 2012, IBM expanded its WebSphere product offerings with Mobile App Dev & Runtime with Worklight…”IBM Mobile Foundation” includes Worklight, Cast Iron, and EndPoint Manager for Mobile dev, runtime, & management.

In the late 90’s, IBM decided to exit the application market and focus on middleware.  The strategy was to recruit business solutions providers (i.e. SAP, Siebel, PeopleSoft) to build on top of IBM MW, and go-to-market as partners. Oracle took a different approach.  Their view was that the application market is fragmented and customers require heavy investments in integrating products and business solutions from different vendors to deliver a complete solution (See Fusion Apps Strategy).  Comparing Oracle’s MW revenue against IBM’s, Oracle’s strategy has worked out better.

Progress

Progress is a mid-size vendor with products & services segmented under Application Development Platform (PureEdge, Orbix), Enterprise Business Solutions (Sonic, Actional, Apama, Savvion) and Enterprise Data Solutions (DataDirect)…

N.B. I reviewed Progress in December of 2012.  Since then, they have fundamentally changed company and product strategies by divesting from integration middleware and focusing on application development (OpenEdge).

Progress makes most of its revenue from its Application Development Platform.  Half of the revenue is realized through indirect channel and OEM.

In 2012, Progress decided to exit the integration middleware business and refocus on application development in Cloud (Application Platform-as-a-Service).  They made a strategic announcement to divest 10 non-core products: Actional, Artix, DataXtend, FuseSource, ObjectStore, Orbacus, Orbix, Savvion, Shadow and Sonic.

Progress sold FuseSource to Red Hat, and Orbix, Artix, and Orbacus to Micro Focus. In addition, it sold off Sonic, Savvion, Actional, and DataXtend to Trilogy enterprises.

Red Hat

Red Hat product lines include infrastructure, middleware, virtualization, Cloud, and storage offerings.  More than 85% of Red Hat’s revenue comes from maintenance and support subscription fees.  Since 2002, Red Hat has had double-digit annual growth.  Between 2005 – 2011, Red Hat’s revenue grew by 319%. In FY2012, Red Hat reached a major milestone of revenues > $1B.

Red Hat’s strategy is continued expansion of middleware offerings, and revenue growth by establishing new routes-to-markets.  They are doing this through investment in OSS, acquisitions, and partnerships.  (Red Hat has a large indirect channel of distributors/resellers, hosting providers, systems integrators, and ISVs.)

Red Hat has an active M&A strategy. With such acquisitions as JBoss, Makara, and FuseSource, it should be on the radar watch for both other open-source vendors such as Novell SUSE, MuleSoft, Talend, … as well as closed-source commercial vendors such as IBM, Oracle, VMware…

Software AG

Software AG is a large, well-established vendor with products & services segmented under Enterprise Transaction Systems (ETS), Business Process Excellence (BPE), and IDS Scheer Consulting (IDSC)…For more information, click here.

Software AG has an impressive revenue growth particularly due to services and consulting.  The acquisition of webMethod gave incremental growth in new licenses and maintenance revenues.  And, the acquisition of IDS Scheer expanded their services and consulting business from 28% of the total revenue in 2005 to almost 40% in 2011.

Product Strategy: App-Server agnostic, SOA, Analytics, Cloud, and Mobile
Growth Strategy: M&A, Geographic Expansion, Strategic Partnerships

Tibco

Tibco is an established integration middleware vendor with a long history in messaging.  Currently, Tibco groups its products along business integration & automation, Event Processing, Analytics (Enterprise, Operational, Big Data), Cloud, and Social.

Despite tough competition from IBM, Oracle with broader mw and infrastructure software portfolio, and open-source integration middleware, Tibco has done a good job of growing revenue in both new licenses and maintenance renewals.  In FY2012, Tibco’s revenue exceeded $1B.

Summary
The middleware software market is split between IBM+Oracle vs. the rest.  Over the last 7 years, smaller vendors like Software AG and Tibco have been able to perform well despite tough competition.  On the open-source software side, Red Hat has been doing very well breaking the $1B revenue mark. Cloud (aPaaS, iPaaS), Big Data, Analytics, and Social are impacting middleware vendors product, service offerings, and go-to-market strategies.

From MVM to Multi-Tenant JVM

Last month, at the JVM Language Summit, Ryan Sciampacone from IBM shared his experience with extending IBM J9 JVM to host multiple applications.  You can view his talk on Oracle Media Network. You can also download a copy of his presentation here. Ryan goes into fairly good details about the challenges and lessons learned, so I recommend taking a look at it.
Across the world in Ireland, Waratek recently announced availability of Waratek Cloud VM for Java.   This JVM extends OpenJDK (HotSpot VM) with a virtualization layer that isolates applications in Java Virtual Containers.  In addition to application isolation, Waratek’s solution offers a container management interface based on Virsh, and resource monitoring… I think this is a very cool solution that could potentially shape ideas to JVM multi-tenancy implementation.  You can watch a presentation on their solution here, and find more information on their solution here. 
-----
Background: The current model of one JVM per application is inefficient.  It leads to JVM sprawls and inefficient usage of infrastructure resources.
The JVM specification defines the JVM as an abstract computing machine. It describes the Class file format,  machine instruction set, rules/constraints on class loading, etc., but contains no requirements or models for the implementation of the JVM’s internal processes, task management or resource management for secure sharing of the JVM by multiple applications. 
Over the last decade, before multi-tenancy had become such a popular (and overused) term, there have been a number of related efforts to address the issues:

• JSR 121: Application Isolation API  – Proposes a language construct called “Isolate” as the means to instantiate isolated Java applications on the JVM.  The spec also addresses isolate-to-isolate communication. 
• JSR 284: Resource Consumption Management (RCM) API – Since there will be different applications sharing a JVM, there needs to be a way to make sure a rogue app does not impact the performance of another app.   JSR 284 proposes a standard API to bind RCM policies to applications running on the JVM.  This JSR is quite interesting, as it has provisions for resource reservation as well setting constraints / quota for resource consumption.

Sun had a couple of reference implementations (MVM, MVM2).  For a complete list of related papers, visit Project Barcelona.) 
SAP also did its own implementation of application isolation scheme by implementing a pool of VMs and a dispatcher to schedule work in the VMs (see Process Attachable Virtual Machines,  Virtual Machine Container).
-----
Java multi-tenancy
There are two usecases for JVM multi-tenancy:

N.B. In the diagram above, I specify App, but App Server should be implied as well.
In option B, you have multiple applications using the JVM.  This is pretty straight forward.  In option A, you have a situation where multiple customers or business units (i.e. shared services) need to share the application securely.
In the case of option A, let’s assume an application has been implemented as a multi-tenant solution using JEE framework.  The App Server interfaces with the underlying JVM to provision new tenants (new isolates).  As part of the tenant provisioning, the App Server uses the RCM interface to set tenant policies.  The JVM monitors the RCM policies, and takes appropriate action.  As application requests hit the App Server, the App Server uses some tenant context scheme to determine which isolate it should go to….As the request flows through the different tiers of the JEE solution, the App Server makes sure only the tenant only accesses the JEE resources that it is authorized to use.  In case of an isolate loop or application runtime environment crash, the JVM applies RCM policies to handle the situation…
In the example above, we can see that there are requirements for the JEE framework as well as JVM runtime to support multi-tenancy.  And, we also should not forget that there re other languages running on the JVM.  Additional requirements may need to be considered.
-----
So, where are we now?
The JCP will standardize multi-tenancy in the Java Platform, but this will probably not happen in JDK8 given the current schedule and the focus for JDK8 (productivity, modularity, performance).
There should be more information available by Jan 2013. 
-----
Final thoughts
With JDK8, there will be a new Java module system (Jigsaw) replacing the old JAR format and class loading from Classpath…  The new format will contain metadata about the packages, classes, and dependencies using a new set of annotations (i.e. version, imports, exports, …), and reflection APIs to improve class loading and sharing, performance,… For some background, have a look at JSR 294 and JSR 277.   This will play an important supporting role in implementing multi-tenancy support in the JVM efficiently.
Clearly, JVM multi-programming / multi-tenancy capabilities will bring economic benefits to both enterprise customers and vendors. It think it might also trigger a new wave of JVM innovations like Waratek.

Enterprise Architects should consider multi-tenancy design patterns & best practices to deliver better shared solutions

Recently, I came across a few good research papers on Cloud and Multi-tenancy design patterns. Typically, the primary audience for such articles is architects and technologists working for service providers (IaaS, PaaS, SaaS, BPaaS…) or software vendors. However, enterprise architects can also apply these architectural principles and patterns to build more cost-effective and easier-to-maintain shared solutions.

In this post, I won’t bore you with an introduction of different Cloud services and taxonomy. I assume you’ve heard it all already. If not, have a look here or just google “Cloud Computing”. My intention is to explain multi-tenancy and key characteristics, review some of the patterns and organize them in logical groups, and provide some references for further examination.

Background
For the last decade, large multi-national enterprises have progressively consolidated IT organizations / functions into Global Shared Services. This is a common strategy to reduce costs by eliminating redundancies, improve operational efficiencies by reducing technology stacks, vendors, and required skills through portfolio rationalization and standardization.

In such organizations, Enterprise Architects are routinely involved in building cost-effective shared solutions. These solutions are used by different business units around the globe requiring BU-specific look & feel, application behavior, and compliance with local customs & regulations.

In the enterprise, a common approach to building shared solutions involves deploying multiple instances of an application or package to accommodate BU-specific functional customizations and non-functional requirements (i.e. load, performance, security…) on a shared infrastructure.

Instances are typically separated in different logical partitions or virtual machines, have their own dedicated resources (application server, middleware, database…), and are configured to meet BU-specific requirements.

Over time, what started out as a common code-base application supporting all BUs diverges into multiple code branches, sometimes so different, that they essentially become different applications. In this case, while infrastructure costs are contained, operational and maintenance complexities and costs grow.

There is a better approach. If applications are designed based on multi-tenancy, the level of “sharedness“ moves higher in the stack. The higher it moves, the better efficiency and asset utilization is gained.

It should be noted that this is appropriate for those organizations that operate at a higher level of IT maturity, as change and release management, and availability and scalability, get more complex.


What is a multi-tenant application?
A multi-tenant application is a shared solution (i.e. CRM) used by different tenants (organizations, BUs). It is a single application with scalable resources to meet the performance demands of tenants.

As each tenant has its own specific functional requirements, the single multi-tenant application can support tenant specific customization at different layers of the architecture (persistence, workflow, component integration, security, UI).

Since multi-tenant applications run on shared infrastructure, it is important to isolate tenants from each other to protect information security, and prevent rogue actions or failure in one tenant’s execution environment does not impact other tenants’ availability.

Finally, since we’re talking about a single code-base running in shared environment, maintenance and updates must be continuously performed without requiring any downtime.

[Note: These are the key concepts behind multi-tenant applications. I am not going to include things like subscription-based licensing or utilization-based billing. Those are in the realm of Software-as-a-Service offering and operation.]

So, the main idea behind multi-tenancy is to provide a single code-base, polymorphic application that is customizable and scalable to meet the demands of different tenants:

In the diagram above, on the left (A) we have a single code-base, single instance, and single DB application supporting different tenants. On the right (B), we have a single code-base and single DB, but multiple instances supporting different tenants. This is not the same as an enterprise deploying multiple instances in their shared solution implementation.

In this case, the reason for “multiple instances” is related to the underlying infrastructure hosting the multi-tenant application. It is related to how its request handling, provisioning, and scalability strategies have been implemented.

In A, there is probably a cluster or grid of compute nodes powering the application. In B, as new application requests arrive, the underlying infrastructure provisions an execution environment (i.e. servlet container) and dispatches the request to the container to run the requests.

Here are some references which you might find useful:
· http://www.computer.org/portal/web/csdl/doi/10.1109/CEC-EEE.2007.4
· https://www6.software.ibm.com/developerworks/offers/techbriefings/cc4d-replays/session2_dcarew.pdf
· http://www.ibm.com/developerworks/webservices/library/ws-multitenantpart2/index.html

Multi-Tenancy Design Patterns & Realization
In this section, I am going to describe some of the key multi-tenancy design patterns and organize them into logical order:

Requirement	Description	Patterns, Approach, and Techniques
Tenant Isolation (Infrastructure)	Tenant isolation is quite broad and multi-dimensional. It is primarily motivated by security and tenant availability and performance.	At the infrastructure layer, tenant execution environment may be separated by virtual machines Higher up, in the application server layer, each tenants may get their own server (Take a look at this WSO2 paper) In the future, OSGi bundles? (See OSGi4C – RFP 133) SOA provides a natural choice for multi-tenant and cloud applications (i.e. reusable components, composite apps). Take a look at this paper by Frank Leymann, et al on combining different multi-tenancy patterns in Service-Oriented applications.
Tenant Isolation (Data)	In a multi-tenant environment, a single database contains data from multiple customers. Tenants should only be able to see their own data.	SQL operations are qualified using tenant_id to return only tenant-specific data. For added security & audit, another field such as user may be required to for authorization and logging. References: http://msdn.microsoft.com/en-us/library/aa479086.aspx
Customization	As previously discussed, the single code-base multi-tenant application must provide some degree of customization and use extension to facilitate adoption. The customizable configuration items range from UI to workflow to database schema. For different layers, different techniques are applied to make the changes. Here is a good reference on approach / methodology including assessing customization needs for a multi-tenant application.	Most SaaS solutions offer a metadata repository and a configuration service. A self-service UI panel is provided to the administrator on the tenant side to make the configuration changes. Wizards are also provided to constrain / guide configuration changes. For most configuration items, change is automatic. On the fly code generation techniques (i.e. templates), and dependency injection are also used to create tenant-specific code at runtime.
Maintenance & Upgrade	Applying patches or system upgrades for a single instance multi-tenant solution running on a shared platform gets quite complex. In the enterprise, change management and release strategies must factor in availability SLAs.	The same principles and best practices as continuous integration and delivery apply here Force.com development lifecycle offers good best practices for release testing and promotion which can also be applied to a multi-tenant solution in the enterprise. Obviously, it is important to be able to roll back to a previous stable point if things go wrong, so here are some general tips: · An on-demand provisioning infrastructure provides an important capability for multi-tenant operation and management. Now, for an enterprise it doesn’t have to get as sophisticated as Amazon or Google, but try to understand the gaps, build a business case, and take appropriate steps. · Similarly, investment in a CMDB solution also plays an important role in developing capabilities to manage change requests effectively and reduce deployment errors and others… · Template-based configuration and deployment…This is an important configuration and change management best practice. · Regular incremental backups (i.e. system snapshots) are also important to enable recovery in the event of rollbacks….

The above have direct linkage to the solutions that are built in enterprise shared services. BUs must be isolated for both technical operational and regulatory reasons. Enterprise architects must find ways to design solutions that share more and cost less in ongoing operation and maintenance. And, the systems typically have high availability requirements.

What’s also important to note is that the there is often no reference model for building shared solutions. This is a big gap. Multi-tenancy patterns and principles can be a big help.

Final thoughts
This is quite a fluid area. Multi-tenancy is still relatively a new concept. The ideas There are a lot of open questions from design -> operation. Enterprise Architects should have this on their radar, as they engage with new projects – or – refactor old implementations.

What do you think?