The IBM zSeries Application Assist Processor, introduced in April 2004 to commemorate the 40th birthday of IBM mainframes, is a speciality assist processor, à la a SAP or ICF, dedicated exclusively to the execution of Java workloads under z/OS(.e). Its goal is to enable integration of new Java based Web applications with core z/OS backend database environments for high performance, reliability, availability, security, and lower total cost of ownership.

It reduces up front capital as well as maintenance costs for workloads with Java cycles; e.g. WebSphere, DB2 etc. It provides additional capacity at low cost of acquisition to process Java without affecting MSU rating or machine model designation.  There is also no additional IBM software costs associated with running a zAAP.

José Castaňo, one of IBM's zAAP technical evangelists, gives us an exclusive, in-depth look at this intriguing, and as yet not very well understood mainframe feature -- in this interview with Anura Guruge, IT In-Depth's editor at large.

José Castaňo: The Bio

Jose has been with IBM since 1986 where he started with TSO/E development, and through 1996 spent his time in various MVS & z/OS components. In 1996 he moved to Parallel Sysplex, first in development and then technical support. He spent 4 years in pre and post sales tech support working with over 100 customers worldwide in helping sales and migrate to Parallel Sysplexes. From there he moved over to Design and Architecture and has been working on various Ease of Use initiatives for z/OS. Next he moved over to e-Business, where he was the strategy manager and helped co-invent the zAAP processor. Jose's current role is Program Director for zSeries strategy, where he manages the strategy department for z/OS and also is the strategy manager for zSeries On Demand initiative. Jose is also the technical chair of the zSeries e-Business Leadership Council, which is a customer council of approximately 40 customers.
 

   Read Anura Guruge's initial take on the zAAP prior to this interview  

 Click here for a January 2005 presentation on zAPP by IBM (which is also available on IBM's portal) [3MB PDF]

Q1: José, can you, at a relatively high-level and somewhat succinctly, give us a “POPs” type description of how a zAAP is optimized to execute Java instructions?

A1: The zAAP runs on a standard processing unit (PU) that can have multiple personalities. A PU can be a standard cpu running zOS, TPF, VSE or zLinux. A PU can be an IFL - which is dedicated to running only zLinux.  A PU can be a SAP which runs the I/O sub-system code.  A PU can be an ICF which runs coupling facility code. With zAAPs a PU is used to run only Java on zOS.  A PU can be a spare in the case where another PU with anyone of the above personalities fails it can swap in and assume the appropriate personality.  The underlying microprocessor and instruction set architecture is the same for all the above personalities.  Modifications to the JVM for zOS detect when Java code is called or when Java code calls out to non-Java code.  These JVM modifications signal the zOS dispatcher than the unit of work can be moved to or from a stand cpu and a zAAP. We've also specialized these engines to avoid some of the common CP function such as taking I/O interrupts.  zAAPs share the L2 cache and main memory with the other PUs on the system. This tight integration of zAAPs with the JVM and the zOS dispatcher on top of the LPAR/PR/SM virtualization technology minimizes latency and cpu degradation while enabling signficant portions of Webshere and java applications to be off-loaded onto engines with substantial price/performance advantages.  This java off-load capability is achieved without any modifications required to the application.  Only the systems programmer will know that the zAAPs are configured.  This memory coherent design provides a number of advantages over network attached distributed offload devices.  It allows us to manage and monitor zAAPs with the same WLM and SMF/RMF infrastructure that is used to manage the other PUs.  It provides the full range of Websphere/zOS, zOS and zSeries QoS that customers expect w/ our platform.

      The zAAP and the proxy for switching into the zAAP is integrated into the JVM, so that we could closely manage the switchover to avoid latency degradation.  We've also integrated this support into the z/OS dispatcher, again for performance.

      As you are probably aware, we continuously improve the performance of JAVA on the platform through path reduction, new architecture in our processors (e.g. TRT) that JAVA can exploit to 'speed loops up', and we also introduced Super-Scalar in our z990s to help JAVA/XML workloads.  However, we've made an development expense decision not limit these enhancements to just the zAAPs, but to make them available to all our processors (CPs, IFLs, zAAPs, etc.) because to have special purpose engines and diverge processors in the same platform is expensive.   So - yes - we do continue to make performance and integration improvements for our JAVA (the recently announced SDK 1.4.2 for z/OS is another example), but we have chosen to make that technology available across the platform, even though in many of the cases, it clearly benefits the JAVA workload more and in some cases it only benefits JAVA workloads.

       This point is very important, because we could have chosen to create different processors or, even another extreme, move JAVA processing to another physical platform.  We chose not to do that, because OLTP and batch workloads, which is the job that z/OS is made for, benefit tremendously from  proximity of Application Serving (and batch) and Data Serving.  The memory coherency of the zAAP and the CPs is very conducive to these workloads and hence the other reason for the decision.

Q2: José, I want to really try and ‘categorize’ what a zAAP can do for a mainframe customer … and whether the main rationale for a zAAP is to ‘off-load’ Java processing away from a standard CP.  So to establish that, can you please walk us through the following scenario.  José, lets say we run the same Java workload, with no other jobs or tasks, on two separate z/OS 1.6 LPARS on a z990, and have one LPAR powered just by a single CP while the other LPAR is powered by a CP and a zAAP.  Will we see a marked performance improvement on the zAAP assisted LPAR … and if so (and I know it is hard to generalize) what would be the “scale” of this improvement?

A2: YES - you would see dramatically improved throughput on the LPAR with 1&1. You are providing additional horsepower with the zAAP, so the work you would be able to accomplish in the zAAP LPAR would be substantially more than the non-zAAP processor.  One would be able to execute the equivalent workload on the 1CP plus 1 zAAP of a 2CP LPAR.

      To elaborate -- if you add a zAAP to a uni processor defined as a standard CP you have close to doubled your overall capacity.  You will see an increase in the capacity of the configuration for a much lower hardware price for the cpu and no increase in software costs relative to upgrading the configuration with a standard CP.  We would expect the increase in the configuration capacity to scale with the the LSPR ratios, assuming there was enough java content in the application to drive the zAAP fully - which in this case would mean that 50% or more of the application is running in Java.  We have a tool that can help a customer understand what the content of their existing java applications are and thus, the potential for off-load to a zAAP.   The overhead associated with switching execution between zAAPs and standard CPs is minimal (less than 5% - we usually see around 1-3% in lab measurements - a recent customer measured 2%) and should not be an inhibitor to realizing the price/performance benefits that zAAPs provide.  Another potential benefit is that by moving the Java processing to the zAAP you can reclaim capacity on the standard CP and use it for some other work.

      There is an exception to the above example.  If you are running on a z890 which is running at a slower than full rated Model 170 speed (we have 7 speed ratings for z890) and you add a zAAP to the configuration - the zAAP will run at the full rated speed of a Model 170 CP while the CP in the same LPAR could be running at a Model 110 speed which is almost 1/14th the speed of the Model 170.  Here is a graphic which illustrates this:

Q3: José, can you share a zAAP between multiple z/OS(.e) 1.6 LPARs?

A3: YES - they can be shared across LPARs, just like CPs, IFLs, and ICFs can be shared.  In is in keeping with our virtualization leadership.

      On a physical server (SMP) you cannot configure more zAAPs than standard CPs. However, within a given LPAR on a server you can define more zAAPs than CPs.

Q4: José, what I am trying to do here is to get to the next level of ‘granularity’ when it comes to ‘killer’ Java apps. for the zAAP.  While we can all appreciate associating the zAAP with obvious Java middleware like WebSphere Application Server, can you, please, get a bit more specific and talk about whether zAAP is really good if you are heavy into Web servers, portal servers, Web caching etc.?

A4: Most Servers either classify themselves as either a data serving or application serving image.  z/OS is different.
We are in the business of OLTP, which is the mix and integration of both data serving and application serving.
This point is critical, because this is where our strengths lie.  To now answer your questions, zAAPs make sense for these type of workloads.   If you require Application Serving that has a strong dependency to the data on z/OS, then these workloads are naturals for zSeries.   If you are doing static web serving,  or heavy portal servers that have very little tie to the zSeries host, then the customer might be better served in either using Linux for zSeries to consolidate those web servers or another platform.    If Web Serving is dynamic and has a tight coupling to EISes or data servers, then this type of WebServing makes sense on z/OS and zAAPs definitely help the case.

      In the lab we have tested zAAPs with a number of workloads with varying degrees of java content.  Here are some examples of the percentage of java content for a range of workloads we tested:

      As applications increase their use of Webservices, WBI-SF workflow and Websphere Portal we would expect that amount of java processing to increase over time.  However, we also see the amount of back-end non-java processing increasing as the scope of the user community increases, new applications are developed and more transactions are driven against the same back-end data source.  It is hard to predict what the relative ratio of java to non-java processing will be in the future, but it is safe to say that there will be more MIPS expended by customers for both.  The more java, the more database and the more tightly the two are integrated the better - this is where zAAPs become attractive.

Q5: José, while I will certainly not put you on the spot and ask for the number per se, but can you please tell us whether it is possible to establish a SPECjjb2000 type rating for a zAAP and if so whether IBM plans to publish such benchmark results at some point?

A5: We do run internal benchmarks and use them to help us measure progress and gaps.   We do NOT publish these benchmarks because we feel that performance benchmarks are very one-dimensional and do not properly reflect what the QOSes and virtualizatoin add to the overall value to the customer and enterprise.

Q6: José, while I know it is possible to have more than one zAAP per LPAR, does it really make sense to have more than one zAAP per LPAR and have more zAAPs per LPAR than standard CPs?

A6: Yes it is possible to have more than one zAAP per LPAR and it definitely makes sense.   The more JAVA workload that is running in the LPAR, then the more zAAP cycles that is required.  For example, if an LPAR is a WAS/DB2, DB2 might require 2 CPs while  WAS might require 2 CPS and 4 zAAPs.   Using the same scenario, it is easy to invisage that one could require more zAAPs than CPs for an LPAR.

Q7: José, this is my last question and I have to ask it though it is OK if you want to plead the “5th” for legal reasons given that I appreciate that this involves your Linux partners … but my question is … if you haven’t already anticipated it … is whether zAAP will, at a future date, be supported by z/Linux?

A7: There are no current plans to support Linux for zSeries.  We will revisit this decision as the market warrants it.

Thank you very much José.

José this was outstanding.  You so eloquently cleared up so many issues and gave us so many valuable insights.  I really appreciate you taking the time to share all this information with us.  I know that our readership will be very grateful for this level of detail.  Best wishes and thank you again.