Internal Tomcat

## 1. Introduction

This document tries to document the design of tomcat-3 and the reasons behind this. Tomcat is the result of a long evolution, with many people with different perspectives contributing code and ideas. Most of the code in tomcat was rewritten 2 or 3 times in the last year, with many experiments and prototypes developed off-line. It is important to distinguish between the actual patterns and ideas and the implementation - the later will change and did changed a lot.

## 2. Goals

- Correctness. Implement the Servlet and JSP specification as accurate as possible.

- Code quality and clarity must match Apache's standards. We also want a bit more comments and documentation :-)

- Security. User applications will run in applet-like sandboxes, and shouldn't be able to get out of the box. Servlet API is the only API ( I am aware of ) that can provide this kind of feature. That means an ISP should be able to let its users to deploy WARs without having to worry about the system security or about the possible interference between the users. It also means companies can use JSPs and applications without having to review them for security flaws ( and in complex web servers this is extremely difficult and expensive ).

- Embedability. The code must be embedable, acting as a servlet container for various applications. There are 2 services tomcat provides - servlet services for Web servers ( like Apache ) and http and servlet services for standalone applications. Tomcat should be usable as a component providing web services for applications ( in the form of the standalone server or in association with a web server), allowing applications to provide servlet-based interfaces. For example it should be possible to create a web-based configuration for an IDE by just starting tomcat and using servlets that access the application's internal APIs. We also want to embed tomcat in existing web servers, providing generic web programming using servlet and JSP for Apache, etc.

- Performance. As in Apache, the performance is less important than code quality and must come from using the right algorithms and data structures, not by low-level hacking. We want to focus on the critical path and be sure we add the smallest overhead possible, while making sure that only applications that make use of special features are paying the price for it.

- Integration with the current Web infrastructure. Web applications will run on the web, and we want to make sure we do can work with existing code and provide a smooth transition. We want to take advantage of the existing optimizations and maturity of web servers like apache, and make sure most of the modules and extensions work well with tomcat.

- Modular structure. The core should be small and as simple as possible ( but not simpler :-), and the most functionality must be provided in modules. By choosing the right modules we can use it in toasters or big servers. Multiple versions of the same module can be optimized for various VMs or special environments.

- Reusable components wherever possible. Functionality that is generic enough should have minimal dependencies on tomcat core. This even more visible starting with tomcat3.3, where we tried to clearly identify the "modules" and build a collection of general-purpose and independent components that can be used in any other project without any relation with tomcat. Having a component that is reviewed, tested and enhanced in more than one project is worth the price of slightly looser integration, and the facade pattern helps integrating independent components without changing their interfaces ( making them dependent on tomcat).

## 3. Design Patterns

### Chain of responsibility

"Avoids coupling the sender of a request to its receiver by giving more than one object a chacne to handle it". In our case the servlet API implementation and the request processing ( mapping, auth, etc) are handler by the Interceptors. The operation is chained and passed along until one interceptor handles it.

Interceptors are the mechanism used to implement and extend tomcat. It avoids coupling the sender of the request ( tomcat core, servlets ) with the actual receiver ( one of the modules handling the requested operation ). The module that actually provide the implementation is not known.

There are 2 important reasons for choosing this pattern for tomcat.

The exact same pattern is used in IIS in the form of filters, NES as SAFs and Apache as modules and hooks. This is the main tool of extending those servers. It also allows the integration between tomcat and web servers ( since the same pattern is used tomcat, will be able to reuse existing code and provide server extensions ). The fact that this works well in existing high performance and the extensibility it provides was a very strong argument.

We also want to integrate tomcat with applications and external packages that may provide various services. This will be a deployment decision, and the core and basic modules shouldn't depend on any particular handler implementation. An example is authentication - the service can be provided by the default tomcat interceptor that use a static xml file, by the native web server, by JAAS, or a private API if tomcat is embedded in applications.

The implementation consists in Interceptors and Events ( possibly in 3.3 or 3.4 ). See also Strategy.

### Strategy

"Define a family of algorithms, encapsulate each one and make them interchangeable".

This allows an easy evolution of tomcat and simplifies the development in an open source environment. We were able to isolate the original implementation of tomcat 3.0 and re-implement individual components while maintaining the overall stability. We started with a code that was very hard to read because of many incremental changes and fixes, but was extremely stable and very well tested. By using this pattern we moved the ugly algorithm in a module and then provided better implementations.

Separating the actual algorithm from the code that calls it allowed us to focus on what was important for the problem, and remove the dependencies.

It allows us to provide alternative implementations for each service ( we have 2-3 authentication modules, 3-4 mapping interceptors were developed and expect more improvements) and decide on a particular implementation based on stability and performance, after the code is ready.

We can choose a set of components that are best tuned for our application - for example we can use a mapping interceptor ( that parses and matches the request against a set of mapping rules ) that is optimized for memory usage or one tuned for speed. An ISP will choose a specialized algorithms that is able to deal with large number of web applications - for example by loading the mappings for a certain application only when needed ( and maintaining a cache of mappings for frequent access)

One particular application is the use of existing algorithms implemented in native servers. Most web servers have highly tuned algorithms ( not a surprise ), and this seems great for lazy programmers - we can get very fast code without too much work.

This is implemented in RequestInterceptors - see also Chain Of Responsibility.

### Observer

This pattern defines a one-to-many dependency between core objects ( Context, ContextManager ) and modules ( Interceptors). When the core object change it's state all the dependents will be notified and updated automatically.

Tomcat state changes at runtime - and all the modules interested in a particular state can be notified and change their internal ( possibly optimized ) data structures. That means the core can be kept simple, without advanced data structures, while modules can use Trees ( JDK1.2 collections) or other complex representations.

It also enable us to keep the core compatible with JDK1.1, while taking advantage of all "modern" features ( this works together with the Strategy pattern in separating the algorithm/data from the caller)

It also allows a simpler core, without any optimization that may affect the code readability. The complex structures are in modules - the Observer will keep the information in sync.

The implementation is done in ContextInterceptor ( that will probably be replaced by a more standard Event/Listener model in 3.3 or 3.4). ContextManager, Context, Container will send notifications each time their state changes, and interceptors will alter their internal state.

### Adapter

It converts the interfaces of a class into the interface tomcat.core expects. This is the main enabler for "integration with other applications" requirement.

Adapter is very important for allowing tomcat to reuse external APIs and modules, without requiring them to change and without changing tomcat ( and making it dependent of a particular API/application). It allows us to use multiple APIs for various tasks - for example authentication can be done using JAAS, J2EE APIs, other proprietary APIs or using the web server. Instead of hard-coding a Realm interface and attempting to fit everything under it ( JAAS is a good candidate for such a "generic auth API ", but it's too much for simple cases ) we just build adapters for whatever API we have to integrate with.

This pattern is also used in integration with the web server, with Request, Response, Container acting as adapters for the equivalent objects in web servers.

This is used in conjunction with "Chain of responsibility" and "Strategy" to enable "lazy programming" ( or reuse of existing APIs :-).

A number of Interceptors are using Adapter for their implementation. J2EEInterceptor is a very good example of adapter - on one side it knows tomcat interface ( Interceptor), and the other side knows J2EE interfaces.

### Proxy

"It provides a "surrogate" or placeholder for another object to control access to it". This is the main tool in isolating the tomcat internals from servlets and user applications. While some form of class loader hacks can also be used to enhance that, this pattern is a very clean mechanism and combined with the Facade provides many other advantages.

It also has an important role in performance, by allowing us to delay expensive operations until they are needed ( String creation ) and use the more efficient interface in core ( see also Facade).

The Servlet API was designed for easy to use and security for application developers, while the constraints and requirements in tomcat.core are very different.

This pattern is used in JDK1.3 to create a type-safe implementation of interfaces ( no down-casting allowed ).

Why? Security was the original reason, but this pattern is at the core of many performance optimizations and will allow a large number of features.

### Facade

Converts the interface of a class into another. In particular, this is used to present web applications with the (right) servlet API, while using more efficient interfaces for the core.

The "facade" name was used ( wrongly ) in many security discussions and in all interfaces, in fact "Proxy" is responsible for security while facade is important for performance and to implement the servlet API on top of a set of internal APIs. For example HttpServletRequest combines access to a number of core interfaces.

Why? The servlet API has certain goals and requirement that are oriented toward the servlet developer. The web server and tomcat.core have different requirements, and by using the right API we can provide a number of optimizations. For example it is possible to avoid String over-use, to use more specialized components, and to use adapters for the (native) web servers.

Another very important benefit of Facade consist in insuring an independence between the current servlet API ( that change with a certain speed ) and the internal representation, and also allows one tomcat instance to support multiple "profiles" of servlet API. For example the transition between servlet 2.2 and 2.3 can be smooth, by allowing 2.2 webapps to run in the same container. This way the users will not have to rewrite ( or recompile ) all their applications every time the spec changes. This can allow tomcat 3.3 to support both servlet 2.2 and servlet 2.3 APIs.

## 4. Main Components ( tomcat.core )

### Request/Response

This is the tomcat representation of a request. It acts as an "Adapter" for the underlying application's representation - in the case of a web server it is the java representation of (request_rec *) for example, if tomcat is embedded in a non-web application it represent the application's notion of request.

This is a passive object, with all operations delegated to modules. It is important to make as much as possible "lazy" evaluations, and keeps the code very simple.

The objects are not a replication of HttpServletRequest and HttpServletResponse. The javax interfaces are designed for web application developers, while core have different needs. It would be very difficult to implement HttpServletRequest as an adapter for request_rec, and it would have a big impact on performance to be restricted to the String-based representation ( that is required for applications - security reasons, etc).

For example, in tomcat 3.2 we started to expose the internal buffers and we'll make more use of this - instead of using the Stream/Writer interfaces, core components will have direct control over the buffering and char/byte translation.

XXX It may be a good idea for the core to use concepts similar with web servers - i.e. only Request, and add Connection to represent the http connection data. The only reason to have Request/Response is to match the servlet model, but the role of core is to match the model used by the web server. A picture would be great.

### OutputBuffer

The idea is to have full control over tomcat buffering and over char/byte translation, as this is one of the most expensive operations ( according to empirical tests ). It should allow to plug external byte/char convertors. As we know, multiple copy from buffer to buffer proved to be an important factor in many systems. It doesn't show up in tomcat 3.2 because other factors are too big, but as we approach the limits it will be one of the most important issue.

An important feature will be the integration of jasper and other template systems, where "static" text can be delivered to the server directly, without intermediary buffers, and the binary representation ( encoding) can be pre-computed.

### MessageBytes

This is a very special and important component of tomcat3. The first problem it tried to solve was the conversion from byte to chars. The charset is known very late ( either when Content-Type header is parsed or even later, when the user sets it in servlet 2.3), and converting the received bytes to Strings can't be done (corectly ) until the encoding is determined.

MessageBytes have another benefit - it's a great mechanism to avoid Garbage Collection costs. Most of the time the user will not access all the headers, and probably neither most of the request information ( url components, etc). That means a "HelloWorld" servlet ( or any servlet that just display some info) will require no String allocations when MessageBytes are used - as oposed to 30-40 Strings allocated ( and GC ) before.

Another ( probably less visible ) benefit is the parsing of the request. Parsing using String proved to lead to inefficient code - Java provides general purpose utilities like StringTokenizer or substring() / indexOf(), but they are designed as "general purpose", with different goals in mind. The parsing code in tomcat 3.0 used those mechanisms and that resulted in 100+ String allocations per request.

Note that String is a very important class in Java - it have the special characteristic of beeing "imutable". That has big implications from a security point of view, and most Servlet API methods are using String as result type/parameter. On the other side String is extremely expensive for server-side operation - it can't be recycled. See also the discussion on GarbageCollection and memory.

### Context

Context is the representation of a web application. It have all the properties that are definable in web.xml and server.xml.

The current implementation is complex and has much more than that - this have to change.

The context is associated with a Request after the contextMap() callback completes. By default the mapping interceptor ( SimpleMapper in the default case ) will implement this hook, but other interceptors can provide extra functionality ( support for ~user, dynamic addition of contexts when the first request is made, etc )

### Container

It represent a group of URLs sharing a common set of properties. It's analog with per_dir configuration in apache. The "standard" properties in servlet spec are the handler ( specified in servlet mappings) and the security properties.

The Container is associated with a Request after the requestMap() hook completes. SimpleMapper is the "core"
implementation of this hook, providing support for prefix, exact and extension mappings. Other interceptors can provide optimized mappings for particular subsets ( like JspIntercepor ) or implement custom mapping schemes.

This is a very important operation ( it should be the most expensive part of the processing - unfortunately we still have to eliminate few other hotspots before this will happen ). We exepect alternate implementations using more advanced data structures and alghoritms ( Trees, etc ). See also the discussion on "evolution".

In tomcat 3.3 it is possible to specify per context and container interceptors - after the mapping, tomcat will know what is the container where the request belongs and can invoke specialized interceptors defined only for that particular set of URLs. This is similar with what all other web servers provide ( the order is similar with NES).

### ContextManager

Main entry point for tomcat execution. It coordinates the activity of modules.

In tomcat, the application embedding tomcat ( web server, generic application, tomcat standalone server) have control over execution. Embedding tomcat requires an adapter that will construct the Request/Response adapters and use ContextManager to process the request.

XXX Probably a better name would be Server, or TomcatContainer.

### Interceptor

The interceptors represents the building blocks and extension mechansim for tomcat. Most of tomcat functionality is implemented using modules. Modules operate on tomcat's core objects and can hook in and extend tomcat. They are designed around "Chain of Responsibility" and "Strategy" patterns, and inspired from Apache hooks, with influences from ISAPI ( filters ) and NSAPI ( saf) . As you know most of Apache ( including the core ) is implemented using a simple hook mechanism.

One interesting sub-project is to make sure interceptors are interoperable with major web server mechansims - and play the same role as mod_perl for example: allow easy server extension using Java ( i.e. provide access to all the full power of native server modules).

You can control all aspects of request processing - parsing, authentication, authorization, sessions, response commit ( before headers are sent), buffer commit ( before any buffer is sent - it can be used to support HTTP1.1 for example !), before and after body.

Interceptors are also used to provide integration with web servers ( in tomcat <=3.2 a special "Connector" interface was used, but we found that Connector was in fact a subset of Interceptor and web integration requires more then just start/stop, so using the same mechanism is a better idea).

Currently tomcat defines a number of hooks - other will be added as needed:

- contextMap

- requestMap

- XXX add the other, descrition

### Helpers ( tomcat.helpers )

In order to keep the core as simple and readable as possible, the few functions that are implemented inside core ( instead of using interceptors ) are implemented in specialized helpers. For example reading the form data, encoding, etc. The helpers are grouped by functionality and the intention is to allow easy evolution - it is possible to rewrite or replace them with minimal effect on the rest of tomcat. This is one way to make sure team development works and new contributors can improve tomcat without the fear of broking the core.

## 5. Execution paths ( ContextManager and core )

### Initialization

XXX server intialization and configuration, context add/init/shutdown/remove, etc

### Request processing

Each web request is received by the web adapter, which will provide the Request/Response adapters. It will then call ContextManager.service(), and tomcat core will begin processing the request. A similar path happens for request that are not generated by a HTTP transaction - like sub-requests or non-web requests ( when tomcat is embedded in applications != web servers )

- Request parsing. Context Manager will delegate ( "Chain of responsibility" ) the parsing to a number of Interceptors, each having a chance to process the request. Tomcat provides a default implementation based on the mappings defined in the spec and static configuration ( server.xml ), and also provide an interceptor that will add all directories in webapp automatically. Other interceptors may provide other features - like automatically adding "~user", etc. See also the performance discussion. It is possible to have different mapping algorithms ( "Strategy" ), including using the web server native mappers if tomcat is integrated with a web server.

- As result of mapping, a number of patterns and rules will be applied to the request and to determine a "Container" where the request belongs. The container encapsulate a set of properties, including the request handler, security constraints, and any other special attributes.

- ContextManager will then delegate the authorization to a set of Interceptors. If a certain role is required, tomcat will call the authentication chain, where an adapter to an authentication API will do the task ( "Strategy" )

- Handler will be called. Expensive/complex operation will be handled by modules ( isUserInRole() for example is handled by the authorization chain ). This is the place where more filtering can be done ( like in Jigsaw or JWS servlet chains ), but so far we haven't found the need.

During request processing, tomcat will send notifications ("Observer") to interested modules about the current stage of the processing, for example allowing the modules to set transactions contexts ( J2EE interceptor ) or instrument tomcat ( timing individual operations for fine tuning ).

This model is identical with the one used in Apache, IIS and NES ( we tried to represent all notifications, so it tries to be a union, not intersection). Why? Performance was the original reason, and the fact that the model was well-known and polished, so more trust-worthy then alternative models. It also turned out that the code organization improved a lot by using the "Strategy" pattern ( compared with tomcat 3.0 ), and many optimizations became possible.

Another reason is related with the idea of making tomcat a real web-server extension, allowing people to develop web server modules ( filter, SAF, hooks) in java using a clean API. There are many ways to implement a web server, but this design choose Apache, IIS and NES as model, and tries to allow bi-directional code reuse.

## 6. Interceptors ( tomcat.context, tomcat.request)

### Standard modules

- SimpleMapper hooks into contextMap and requestMap and implements the standard mapping rules defined in the servlet API. It is not a particulary fast module, but it's well tested and stable. Most of the current code is derived and can be traced back to tomcat 3.0. We hope that other solutions will show up and will evolve to replace SimpleMapper.

- LoaderInterceptor will set the context ClassLoader. There are 2 modules - one that provide a custom class loader to be used with JDK1.1 and one that plugs JDK1.2 URLClassLoader.

### Adding custom modules

### Revolution by Evolution

By using the "Strategy" pattern it is possible to provide alternative alghoritms and implementations without affecting the core modules. In time the new "strategies" can be tested ( by first using them on experimental sytes, then in production ) and eventually replace the "core" modules.

Tomcat is designed such that the core will have a minimal role in how the request is processed. We try to keep the number of methods and structures as small as possible, while "evolving" the core based on the use cases and needs of various modules. We expect that 3.3 or 3.4 will have a stable core.

This method allowed a smooth evolution from tomcat3.0 who had a very complex and ugly (!) code, due to numerous additions and fixes. It is a standard form of refactoring that proved very useful.

## 7. Reusable Components ( tomcat.util )

Tomcat tries as much as possible to keep most of the code in standalone components, that have no dependency on tomcat.core. For example, the class loader, the session manager, the TCP server, authentication - all of them can easily be used in different architectures, and don't (shouldn't ) have any reference to core interfaces.

There are many reasons for this: we think tomcat should focus on implementing the servlet API and shouldn't reinvent the wheel, and we also think all those components should be generic enough to be used in other servers.

For example, most servlet container allows you to plug in a session manager. Session manager have ( shouldn't have ) too much need for knowing about the container internals - it is a special form of object database. Most of the time is better to write code independent of a particular container and using adapters to hook into different containers.

### ThreadPool

## 8. Security

XXX Move to separate document!

The Proxy and FacadeManager plays a very important role in insuring security, by providing control of the communication between webapps and tomcat internals. It is possible to augment that with a special class loader mechanism ( for paranoid administrators), but most of the information that is available suggest the proxy is a very effective design pattern for this.

Since tomcat.core runs with special (java) privileges, it is important to make sure the code is safe and the user can't call arbitrary methods. That will require a serious review over what methods are public, and how internal object instances can be accessed from outside. FacadeManager should be the only point of entry.

We do need to provide a certain internal API to applications that will allow enhanced performance - for example Jasper can make use of the internal buffers, etc.

### Existing risks and problems

Isolation. It is possbile for an application to store the reference to the HttpServletRequests it receives in a Hashtable. If the server is using recycling, the "bad" application will end up with references to all active requests, and will be able to access parameters and informations of other web applications.
  In tomcat the HttpServletRequest is a proxy and a facade for the real request. Tomcat will recycle the Request object, and can ( based on a config option) create new HttpServletRequestFacades for each request.
Constructor access. It is possible that an application will use the current internal APIs to gain unauthorized access and call internal APIs
  The only way to get access to internal APIs is using a special context/request attribute. We allow only "trusted" applications to call the method, and return null if the trusted flag is not set. We need to carefully review the code to be sure no other way to access the internals is possible, and reduce the number of public constructors and public methods. We also need to add a ( JDK1.1 compatible ) mechanism to do security checks when objects are constructed. ( for example add a TomcatSecurity object with empty methods for 1.1 and using TomcatPermission and the 1.2 security system for 1.2.
DOS - large body with POST data
  Tomcat will read the full body when a POST is processed. We need to set upper limits or at least warn of possible abuse.
DOS - general
  There are many possible DOS attacks, we need to identify them and provide mechanisms to reduce the effect.

## 9. Embedding tomcat

Use the tomcat APIs to create ContextManager and set it up or stop it. Integration with your configuration API is very simple - tomcat doesn't care how you set it up, as long as the right object are created and the setter methods are called.

All integration is done using the "Adapter" pattern - the Interceptors provide you with hooks into all tomcat internals, and you can control every aspect of request processing.

## 10. Integration with Web servers

Most commercial web sites run on one of the 4 big web servers, and we want to make sure tomcat can work well with those. After all, most people are using servlets for web applications :-)

There are huge problems, even with the current architecture. For example the mapping defined in servlet 2.2 is different from the mapping used in the current web servers, form based login is very difficult to integrate ( and will require a lot of native code to protect non-java resources using the same authentication ).

Most of the problems still require a lot of work, but we hope that the current architecture will be able to handle that. A bigger goal will be to transform the Interceptors into real web server extensions, so people can develop hooks/modules/SAF/filters in java instead of C ( or mod_perl ).

### Tomcat and other Web Servers

Tomcat design is based on the features of Apache modules/hooks, ISAPI and NSAPI. We tried to not limit ourself to what is common to all. In time it is likely that tomcat will have more hooks and provide the same level of extensibility as the other servers. We tried to add the hooks by following "when somebody needs it " rule - and so far a number of callbacks are still to be added.

Please read the Apache, IIS and NES documentations & code, as. Except for using different names and Java-style for interfaces, the model is the same.

The mapping is:

Tomcat | Apache | ISAPI | NSAPI | Other
--- | --- | --- | --- | ---
Request / Response | request_rec conn_rec |  |  |
ServletWrapper | Handler |  |  |
Context | server_rec / per_dir_config |  |  |
Container | per_dir_config |  |  |
Interceptor | Module / hooks |  |  |
ContextManager |  |  |  |

Missing:

- pictures/UML diagrams

- policy-based security - it's standard, but need to add more info about the implementation

- Relation with JDK1.2 / JDK1.1 - based on the thread on tomcat-dev

- Current RequestMapper arch and future - based on the mails from carlos@aper.net

- reloading/class loaders - I'm working on cleaning up the implementation, I'll add documentation after that

- Request notes - I have to clean up the actual design and implementation first, make it a standalone utility

- Real numbers - this will be slow and change as we implement tomcat3.3. We'll have a complete set before October ( ApacheCon). We need to keep track of the effect on performance of all important changes we do, and also the overhead.

- In-detail documentation for all stand-alone components. Those are un-likely to change too much and is important to explain they are independent of tomcat.

- In particular, XMLMapper and the configuration, mentions of JNDI configuration.

- Module life cycle + picture

- The authentication module.

- Integration with apache - config generation,etc - unfortunately we don't know yet very well how to do it.