Dispatcher

The Dispatcher is Adobe's caching and/or load balancing tool. This section aims to help you:

• Configure the Dispatcher for efficient operation.
  
• Design your website to optimize operation of the Dispatcher when creating a high-performance website.
• Fine-tune your Dispatcher settings should performance issues arise.

Using the Dispatcher also helps protect your application server from attack. Therefore, you can increase protection of your CQ instance by using the Dispatcher in conjunction with an industry-strength web server.

The process for deploying the Dispatcher is independent of the web server and OS platform chosen:

• Install the supported web server of your choice according to their own documentation.
  
• Install the Dispatcher module appropriate to the chosen web server and configure the web server accordingly.
• Configure the Dispatcher.
  
• Integrate with CQ to update the cache when the content in CQ changes.

There are two basic approaches to web publishing:

• Static Web Servers: such as Apache or IIS, are very simple, but fast.
• Content Management Servers: which provide dynamic, real-time, intelligent content, but require much more computation time and other resources.

The Dispatcher helps realize an environment that is both fast and dynamic. It works as part of a static HTML server, such as Apache, with the aim of:

Which means that:

• static content is handled with exactly the same speed and ease as on a static web server;additionally you can use the administration and security tools available for your static web server(s).
• dynamic content is generated as needed, without slowing the system down any more than absolutely necessary.

The Dispatcher contains mechanisms to generate, and update, static HTML based on the content of the dynamic site. You can specify in detail which documents are stored as static files and which are always generated dynamically.

This section illustrates the principles behind this.

A static web server, such as Apache or IIS, serves static HTML files to visitors of your website. Static pages are created once, so the same content will be delivered for each request.

This process is very simple, and thus extremely efficient. If a visitor requests a file (e.g. a HTML page), the file is usually taken directly from memory, at worst it is read from the local drive. Static web servers have been available for quite some time, so there is a wide range of tools for administration and security management, and they are very well integrated with network infrastructures.

If you use a Content Management Server, such as CQ, an advanced layout engine processes the request from a visitor. The engine reads content from a repository which, combined with styles, formats and access rights, transforms the content into a document that is tailored to the visitor's needs and rights.

This allows you to create richer, dynamic content, which increases the flexibility and functionality of your website. However, the layout engine requires more processing power than a static server, so this setup may be prone to slowdown if many visitors use the system.

Methods for Caching

The Dispatcher has two primary methods for updating the cache content when changes are made to the website.

• Content Updates remove the pages that have changed, as well as files that are directly associated with them.
• Auto-Invalidation automatically invalidates those parts of the cache that may be out of date after an update. i.e. it effectively flags relevant pages as being out of date, without deleting anything.

Content Updates 

In a content update, one or more CQ documents change. CQ sends a syndication request to the Dispatcher, which updates the cache accordingly:

1. It deletes the modified file(s) from the cache.
2. It deletes all files that start with the same handle from the cache. For example, if the file /en/index.html is updated, all the files that start with /en/index. are deleted. This mechanism allows you to design cache-efficient sites, especially in regard to picture navigations.
3. It touches the so-called statfile; this updates the timestamp of the statfile to indicate the date of the last change.

The following points should be noted:

• Content Updates are typically used in conjunction with an authoring system which "knows" what must be replaced.
• Files that are affected by a content update are removed, but not replaced immediately. The next time such a file is requested, the Dispatcher fetches the new file from the CQ instance and places it in the cache, thereby overwriting the old content.
• Typically, automatically generated pictures that incorporate text from a page are stored in picture files starting with the same handle - thus ensuring that the association exists for deletion. For example, you may store the title text of the page mypage.html as the picture mypage.titlePicture.gif in the same folder. This way the picture is automatically deleted from the cache each time the page is updated, so you can be sure that the picture always reflects the current version of the page.
• You may have several statfiles, for example one per language folder. If a page is updated, CQ looks for the next parent folder containing a statfile, and touches that file.

Auto-invalidation

Auto-invalidation automatically invalidates parts of the cache - without physically deleting any files. At every content update, the so-called statfile is touched, so its timestamp reflects the last content update.

The Dispatcher has a list of files that are subject to auto-invalidation. When a document from that list is requested, the Dispatcher compares the date of the cached document with the timestamp of the statfile:

• if the cached document is newer, the Dispatcher returns it.
• if it is older, the Dispatcher retrieves the current version from the CQ instance.

Again, certain points should be noted:

• Auto invalidation is typically used when the inter-relations are complex e.g. for HTML pages. These pages contain links and navigation entries, so they usually have to be updated after a content update. If you have automatically generated PDF or picture files, you may choose to auto-invalidate those too.
• Auto-invalidation does not involve any action by the dispatcher at update time, except for touching the statfile. However, touching the statfile automatically renders the cache content obsolete, without physically removing it from the cache.

Finding out whether a document is subject to caching

You can define which documents the Dispatcher caches in the configuration file. The Dispatcher checks the request against the list of cacheable documents. If the document is not in this list, the Dispatcher requests the document from the CQ instance.

The Dispatcher always requests the document directly from the CQ instance in the following cases:

• If the HTTP method is not GET. Other common methods are POST for form data and HEAD for the HTTP header.
• If the request URI contains a question mark "?". This usually indicates a dynamic page, such as a search result, which does not need to be cached.
• The file extension is missing. The web server needs the extension to determine the document type (the MIME-type).
• The authentication header is set (this can be configured)

Finding out if a document is cached

The Dispatcher stores the cached files on the web server as if they were part of a static website. If a user requests a cacheable document the Dispatcher checks whether that document exists in the web server's file system:

• if so, it returns this
• if not, the Dispatcher requests the document from the CQ instance.

Finding out if a document is up-to-date

To find out if a document is up to date, the Dispatcher performs two steps:

1. It checks whether the document is subject to auto-invalidation. If not, the document is considered up-to-date.
2. If the document is configured for auto-invalidation, the Dispatcher checks whether it is older or newer than the last change available. If it is older, the Dispatcher requests the current version from the CQ instance and replaces the version in the cache.

Load Balancing is the practice of distributing the computational load of the website across several instances of CQ.

You gain:

• increased processing power
  In practice this means that the Dispatcher shares document requests between several instances of CQ. Because each instance now has fewer documents to process, you have faster response times. The Dispatcher keeps internal statistics for each document category, so it can estimate the load and distribute the queries efficiently.
• increased fail-safe coverage
  If the Dispatcher does not receive responses from an instance, it will automatically relay requests to one of the other instance(s). Thus, if an instance becomes unavailable, the only effect is a slowdown of the site, proportionate to the computational power lost. However, all services will continue.
• you can also manage different websites on the same static web server.

Performance Statistics

The Dispatcher keeps internal statistics about how fast each instance of CQ processes documents. Based on this data, the Dispatcher estimates which instance will provide the quickest response time when answering a request, and so it reserves the necessary computation time on that instance.

Different types of requests may have differing average completion times, so the Dispatcher allows you to specify document categories. These are then considered when computing the time estimates. For example, you can make a distinction between HTML pages and images, as the typical response times may well differ.

If you use an elaborate search function, you may create a new category for search queries. This helps the Dispatcher send search queries to the instance that responds fastest. This prevents a slower instance from stalling when it receives several "expensive" search queries, while the others get the "cheaper" requests.

Personalized content (Sticky Connections)

Sticky connections ensure that documents for one user are all composed on the same instance of CQ. This is important if you use personalized pages and session data. The data is stored on the instance, so subsequent requests from the same user must return to that instance or the data is lost.

Because sticky connections restrict the Dispatcher's ability to optimize the requests, you should use them only when needed. You can specify the folder that contains the "sticky" documents, thus ensuring all documents in that folder are composed on the same instance for each user.

Each Dispatcher installation kit comes as an archive file that can be downloaded from Daycare.

The following naming convention is used:

dispatcher-<web-server>-<operating-system>-<dispatcher-release-number>.<file-format>

For example, the dispatcher-apache2.2-linux-i686-4.0.6.tgz installation kit contains the dispatcher release 4.0.6 for an Apache 2.2 web server that runs under Linux i686 and is packaged using the tar format.

The naming according to web server is as follows:

Each archive contains the following files:

• the Dispatcher modules
  
• an example configuration file
• the readme file with installation instructions and last-minute information; the readme file is named as README.dispatcher.<web-server-name> e.g. README.dispatcher.apache
• release notes

The following sections detail the web server specific installation procedures.

For information on how to install this web server, see:

• Microsoft's own documentation on the Internet Information Server
• "The Official Microsoft IIS site"

IIS 7 / 7.5

After installation:

1. Open the Server Manager and add the role Web Server (IIS).
2. Ensure the following Role Services are installed:
• ISAPI Extensions
  
• ISAPI Filters
  
• IIS 6 Metabase Compatibility

See Installation for information on accessing the installation files. The required archive for Microsoft Internet Information System is:

• dispatcher-iis-<operating-system>-<dispatcher-release-number>.zip

which contains the following files :

IIS 5 / 6

1. Extract
   
   • disp_iis.dll
   • disp_iis.ini
   • dispatcher.any
   • disp_iis.pdb
   from the Dispatch package into the executable directory of the selected website under IIS; 
   i.e. <IIS_INSTALLDIR>\scripts. For example, in a standard IIS installation on a non-server product, the installation directory is C:\Inetpub, with the scripts virtual directory located in C:\Inetpub\Scripts.
   

IIS 7 / 7.5

To add the Dispatcher to the list of available ISAPI filters use the following steps:

1. Using the Windows Explorer, create a directory <IIS_INSTALLDIR>/Scripts;
   for example, C:\Inetpub\Scripts.
2. Extract all files from the scripts directory of the Dispatcher package into this directory:
   • disp_iis.dll
   • disp_iis.ini
   • dispatcher.any
   • disp_iis.pdb

1. Configure disp_iis.ini as required. The basic format of the .ini file is as follows:

An example configuration:

IIS 5 / 6

To add the Dispatcher to the list of available ISAPI filters use the following steps:

1. Inside the Internet Service Manager, right click the root node of the website under which you want to add the dispatcher, then open its Properties dialog.
2. Select the tab named ISAPI Filters.
3. Click Add.. and specify:
   • Filter Name, a name for the filter (i.e. for the Dispatcher).
   • Executable, the location of the filter (i.e. where the dll resides).
4. Click OK to save.

Access to the cached files must be unrestricted (Anonymous Access). Otherwise the syndication request for cache flushing will be unsuccessful due to missing authentication information.

To ensure access you have to:

1. Inside the Internet Service Manager, right click the root node of the appropriate website, then open its Properties dialog.
   
2. Select the Directory Security tab.
3. Activate Anonymous access.
4. To activate the changes you have to restart IIS. Either from the IIS control window or from a command window:
   
   • net stop w3svc  - will stop the IIS web publishing service
   • net start w3svc - will start it again

IIS 7 / 7.5

To add the Dispatcher to the list of available ISAPI filters perform the following tasks.

Configure the Virtual Directory:

1. Open Administrative Tools, then select Internet Information Services (IIS) Manager.
2. Select your site in the tree, then using the context menu (usually right mouse click) select Add Virtual Directory....
3. Enter the alias Scripts and the physical path of the directory created above (C:\Inetpub\Scripts).

Register the ISAPI filter:

1. Open Administrative Tools, then select Internet Information Services (IIS) Manager.
2. Select your site in the tree, then the tab Features View (at the bottom).
3. Open the feature ISAPI Filters in the view.
4. Click Add... and enter the following settings:
   • Filter Name: CQ
   • Executable: the path to disp_iis.dll (C:\Inetpub\Scripts)
5. Click OK to save.

Register the ISAPI handler:

1. Open Administrative Tools, then select Internet Information Services (IIS) Manager.
2. Select your site in the tree, then the tab Features View (at the bottom).
3. Open the feature Handler Mappings in the view.
4. Click Add Script Map... and enter the following settings:
   • Request Path: /Scripts/disp_iis.dll
   • Executable: the path to disp_iis.dll (C:\Inetpub\Scripts)
   • Name: CQ
5. Click OK to save.
6. Open the feature Configuration Editor.
7. Select the section system.webServer\handlers from Application.config.
8. Select the first Collection (at the top), then click on ... (at the far right).
9. The Collection Editor will appear, select the handler CQ (at the top).
10. Change the value of the allowPathInfo flag (at the bottom) to true.
    
11. Close the Collection Editor and click Apply.

Register the .json extension:

1. Open Administrative Tools, then select Internet Information Services (IIS) Manager.
2. Select your site in the tree, then the tab Features View (at the bottom).
3. Open the feature MIME Types in the view.
4. Click Add... and enter the following settings:
   • File name extension: .json
   • MIME type: application/json
5. Click OK to save.

IIS 7.5

If your site is not completely new it might contain the hidden segment bin. To remove this:

1. Open Administrative Tools, then select Internet Information Services (IIS) Manager.
2. Select your site in the tree, then the tab Features View (at the bottom).
3. Open the feature Request filtering in the view.
4. Select the tab Hidden Segments.
5. Remove the entry bin (if existing).

If you would like the dispatcher to write a log file instead of writing to the event log then do the following:

1. First, configure a logging location in disp_iis.ini. Open the disp_iis.ini file for editing, this file is most likely stored under C:\inetpub\scripts\disp_iis.ini.
   
2. Add the following value:
   logfile=C:\inetpub\logs\dispatcher\dispatcher.log
3. Using Windows Explorer, go to C:\inetpub\logs directory.
4. Create a directory named dispatcher
   
5. Right click the dispatcher directory and select Properties in the contextual dialog
6. Select the Security tab
7. Click Edit
   
8. Click Add
9. Open Administrative Tools, then select Internet Information Services (IIS) Manager.
10. In the IIS Manager, browse down the tree and select the IIS site where you have configured the dispatcher. On the right side of the window, click Advanced Settings.
11. Highlight and copy the value under Application Pool (copy by pressing [Ctrl]+c).
    
12. Go back to the security Add dialog.
13. In the editable box, enter IIS AppPool\ then paste the value from step 10. You should have entered something like this IIS AppPool\DefaultAppPool.
    
14. Click Check Names (this should cause the user id to be underlined).
15. Click OK.
16. For the newly added user, enable all access rights except Full Control.
17. Click OK.

Before you can start using the Dispatcher you must now:

• Configure the Dispatcher
• Integrate it with any CQ instances involved.

For Information about how to install an Apache Web Server read the installation manual - either online or in the distribution.

The Dispatcher comes as either:

• Windows: a Dynamic Link Library (DLL)
• Unix: a Dynamic Shared Object (DSO)

See Installation for information on accessing the installation archive files, in particular the specific file to select for your environment.

The installation archive files contains the following files - dependent on whether you have selected Windows or Unix:

Use the following steps to add the Dispatcher to your Apache Web Server:

1. Place the Dispatcher file in the appropriate Apache module directory:
• Windows:
  Place disp_apache<x.y>.dll in <APACHE_ROOT>/modules
• Unix:
  Locate either the <APACHE_ROOT>/libexec or <APACHE_ROOT>/modules directory according to your installation.
  Copy dispatcher-apache<options>.so into this directory.
  To simplify long-term maintenance you can also create a symbolic link named mod_dispatcher.so to the Dispatcher:
      ln -s dispatcher-apache<x>-<os>-<rel-nr>.so mod_dispatcher.so

The Apache Web Server needs to be configured, using httpd.conf. In the Dispatcher installation kit you will find an example configuration file named httpd.conf.disp<x>.

These steps are compulsory:

1. Navigate to <APACHE_ROOT>/conf.
2. Open httpd.conf for editing.
   
3. The following configuration entries must be added, in the order listed:
   • LoadModule to load the module on start up.
   • AddModule to enable the module. (Apache 1.3 only).
     
   • Dispatcher-specific configuration entries, including DispatcherConfig,DispatcherLog and DispatcherLogLevel.
   • SetHandler to activate the Dispatcher. LoadModule.
   • ModMimeUsePathInfo to configure behavior of mod_mime.

The following configuration steps are optional, but recommended:

1. Change the owner of the htdocs directory:
   • The apache server starts as root, though the child processes start as daemon (for security purposes). The DocumentRoot (<APACHE_ROOT>/htdocs) must belong to the user daemon:
         cd <APACHE_ROOT>
         chown -R daemon:daemon htdocs
2. For Apache 1.3 only:
   Remove the Multiviews option for directories handled by the dispatcher.
   • When a file is requested and its parent directory does not yet exist, the negotation module returns a 403 (FORBIDDEN) response before the dispatcher has the opportunity to create the file and its parent directories. Therefore, you should disable the MultiViews option inside directories that are handled by the dispatcher.
     To do this, remove the following lines in httpd.conf:
         Options Indexes FollowSymLinks
         Options Indexes MultiViews

LoadModule

The following table lists examples that can be used; the exact entries are according to your specific Apache Web Server:

AddModule (Apache 1.3 only)

A list of AddModule statements enable modules within the Apache Web Server.

The order of AddModule statements in the configuration file is important if there are interdependencies between the modules. It is recommended to position the AddModule statement for the Dispatcher as the last entry; for example:

AddModule disp_apache.c

Dispatcher specific configuration entries

The Dispatcher-specific configuration entries are placed after the LoadModule entry. The following table lists an example configuration that is applicable for both Unix and Windows:

The individual configuration parameters:

SetHandler

After these entries you must add a SetHandler statement to the context of your configuration (<Directory>, <Location>) for the Dispatcher to handle the incoming requests. The following example configures the Dispatcher to handle requests for the complete website:

The following example configures the Dispatcher to handle requests for a virtual domain:

ModMimeUsePathInfo

After the SetHandler statement you should also add the ModMimeUsePathInfo definition.

The ModMimeUsePathInfo parameter should be set On for all Apache configurations:

    ModMimeUsePathIfo On

The mod_mime module (see for example, Apache Module mod_mime) is used to assign content metadata to the content selected for an HTTP response. The default setup means that when mod_mime determines the content type, only the part of the URL that maps to a file or directory will be considered.

When On, the ModMimeUsePathInfo parameter specifies that mod_mime is to determine the content type based on the complete URL; this means that virtual resources will have metainformation applied based on their extension.

The following example activates ModMimeUsePathInfo:

Before you can start using the Dispatcher you must:

• Configure the Dispatcher
• Integrate it with any CQ instances involved

For full information on how to install these web servers, please refer to their respective documentation:

• Sun Java System Web Server
• iPlanet Web Server

The Dispatcher comes as either:

• Windows: a Dynamic Link Library (DLL)
• Unix: a Dynamic Shared Object (DSO)

See Installation for information on accessing the installation files; in particular the specific file to select for your environment.

The installation archive files contains the following files - dependent on whether you have selected Windows or Unix:

Use the following steps to add the Dispatcher to your web server:

1. Place the Dispatcher file in the web server's plugin directory:
• Windows:
  Place disp-ns.dll in the directory <WEBSERVER_ROOT>/plugins.
• Unix:
  Place dispatcher.so in the directory <WEBSERVER_ROOT>/plugins.
  

The web server needs to be configured, using obj.conf. In the Dispatcher installation kit you will find an example configuration file named obj.conf.disp.

1. Navigate to <WEBSERVER_ROOT>/config.
2. Open obj.conf for editing.
3. Copy the line that starts:
       Service fn="dispService"
   from obj.conf.disp to the initialization section of obj.conf.
   
4. Save the changes.
5. Open magnus.conf for editing.
6. Copy the two lines that start:
     Init funcs="dispService, dispInit"
     and
     Init fn="dispInit"
   from obj.conf.disp to the initialization section of magnus.conf.
7. Save the changes.

Init

The following table lists examples that can be used; the exact entries are according to your specific web server:

Depending on your requirements you can define the Dispatcher as a service for your objects. To configure the Dispatcher for your entire website modify the default object:

Before you can start using the Dispatcher you must:

• Configure the Dispatcher
• Integrate it with any CQ instances involved

The following sections describe how to configure various aspects of the Dispatcher.

By default the Dispatcher configuration is stored in dispatcher.any, though you can change the name and location of this file during installation.

You can also include files:

• If your configuration file is large you can split it into several smaller files (that are easier to manage) then include these. 
  
• To include files that are generated automatically.

For example, to include the file myFarm.any in the /farms configuration use the following code:

You can also use the asterisk ("*") as a wildcard to specify a range of files to include.

For example, if the files farm_1.any through to farm_5.any contain the configuration of farms one to five, you can include them as follows:

One dispatcher can be used with multiple websites; for example:

• intranet.myCompany.com
• internet.myCompany.com
• www.myFlagshipProduct.com

To configure your Dispatcher you:

• assign a unique /name
• configure the caching and rending parameters for each /farm
• configure logging

An example configuration is structured as follows:

With the /name parameter you assign a unique name (of your choice) to your Dispatcher instance.

Any file system oriented system call can be interrupted EINTR if the object of the system call is located on a remote system accessed via NFS. Whether these system calls can time out or be interrupted is based on how the underlying file system was mounted on the local machine.

The parameter can be used if your instance has such a configuration and you have seen the following log message:

Error while reading response: Interrupted system call

Internally the the dispatcher reads the response from the remote server (i.e. CQ) using a loop that can be represented as:

while (response not finished) {
     read more data
}

Such messages can be generated when the EINTR occurs in the "read more data" section and are caused by the reception of a signal before any data was received.

To ignore such interrupts you can add the following parameter to dispatcher.any (before /farms):

/ignoreEINTR "1"

Setting /ignoreEINTR to "1" will instruct the dispatcher to stay in the loop for reading data until the complete response has been read. The default value is 0 and deactivates the option.

The /farms section defines a list of farms or websites; they can have any alphanumeric (a-z, 0-9) name.

Each /farms section defines:

• A set of load-balanced renderers.
• The IP addresses and ports of the publish instances to serve and cache content from.
• Further characteristics including where to cache files, what to cache.

For each farm you can specify separate caching and rendering parameters, some of which have sub-parameters:

The following example shows the skeleton definition for two farms named /daycom and /docsdaycom:

If you use more than one render farm, the list is evaluated bottom-up. This is particularly relevant when defining Virtual Hosts for your websites.

This specifies the page that the Dispatcher returns when no specific target page or file is requested.

Typically this is the page returned when a user specifies an URL such as www.myCompany.com. The /homepage parameter is required if there is no automatic redirection from the server (for example IIS) nor from CQ (for example, if you shut CQ down after the content is cached). To display the index.html page in such circumstances, use the following setting:

This is defined within the /farms section, for example:

This defines a list of all HTTP headers passed from the client to the CQ instance.

By default the Dispatcher forwards the standard HTTP headers to the CQ instance. In some instances, you might want to:

• add headers: for example, custom headers
• remove headers: for example, authentication headers which are only relevant to the web server

If you need to customize the set of headers you have to specify the entire set of headers to be forwarded; including any of those forwarded by default. Such a list might look as follows:

The virtual hosts section is a list of all hostname/URI combinations that the Dispatcher accepts for this website. You can also use the asterisk ("*") character as a wildcard. For example, the section:

handles the requests for:

• myCompany - both the .com and the .ch domains
• mySubDivision - all domains

To configure the Dispatcher to handle all requests:

If you use more than one render farm, the list is evaluated bottom-up.  In the following example (which shows only the relevant sections):

• all the pages in the /products folder are sent to server 2
• all other pages to server 1

This feature allows you to create a secure session for access to the render farm so that users:

• need to log in before they can access any page in the farm
• have access to all pages in the farm after logging in

/sessionmanagement is defined within /farms.

/sessionmanagement has several sub-parameters:

/directory (mandatory)

The directory that stores the session information. If the directory does not exist, it is created.

/encode (optional)

How the session information is encoded. Use "md5" for encryption using the md5 algorithm, or "hex" for hexadecimal encoding. If you encrypt the session data, a user with access to the file system cannot read the session contents. The default is "md5".

/header (optional)

The name of the HTTP header or cookie that stores the authorization information. If you store the information in the http header, use HTTP:<header-name>. To store the information in a cookie, use Cookie:<header-name>. If you do not specify a value HTTP:authorization is used.

/timeout (optional)

The number of seconds until the session times out after it has been used last. If not specified "800" is used, so the session times out a little over 13 minutes after the last request of the user.

An example configuration looks as follows:

This section defines where the Dispatcher will allocate requests to render a document.

If you use a single CQ instance for rendering, you can specify it as in the following example:

If the CQ instance runs on the same computer as the Dispatcher, you can specify it as in the following example:

To distribute the workload equally among multiple CQ instances use:

Using filters, you can specify which requests are accepted by the Dispatcher module. All other requests are sent back to the server, where they are offered to the other modules that run on the web server. 

If you want the Dispatcher to handle all files, use:

Filters also allow you to exclude (deny) access to various elements; for example:

• ASP pages
• sensitive areas within a publish instance

Requests to an explicitly denied area result in a 404 error code (page not found) being returned.

The following filter denies access to ASP pages:

A match is needed for the entire request line, not just the URI. Therefore the above defines the match as "*.asp *", because the full request line is GET /home.asp HTTP/1.0.

You can also match other parts of the request. The following filter denies access to form data submitted by the POST method:

The following example shows a filter used to deny external access to the Workflow console:

If your publish instance uses a web application context (for example publish) this can also be added to your filter definition.

If you still need to access single pages within the restricted area, you can allow access to them. For example, to allow access to the Archive tab within the Workflow console add the following section after the previous example:

Syndication requests are usually intended for the Dispatcher only, so by default they are not sent to the CQ instance.

If necessary, you can forward syndication requests to the Dispatcher. This is done by setting this parameter to "1". If set, you must make sure that POST requests are not denied in the filter section.

The /cache section specifies some general aspects of how and where the Dispatcher caches documents.

Various sub-parameters can be defined:

/docroot

This link points to the document root of the web server. This is where the Dispatcher stores the cached documents, and this is where the web server looks for them. If you use multiple render farms, you have to define a different document root on the web server for each farm, and specify the corresponding link here.

/statfile

This link points to the statfile, which the Dispatcher uses to keep track of the last content update. This can be any file on the web server. The file itself is empty, only the timestamp is used.

/statfileslevel

Creates statfiles in all folders down to the level you specify. Use this if you only want to invalidate sections of the cache, not the entire cache. For a default structure with language folders (for example, the English site under /content/myWebsite/en), use /statfileslevel = "2" to invalidate per language. This means that when the Dispatcher invalidates the English part of the website, other sections are not affected (for example, the German and French sections). If you use this parameter, do not specify the /statfile parameter.

/allowAuthorized

By default, requests that carry an authentication header are not cached. This is because authentication is not checked when a document is returned from the cache - so the document would be displayed for a user who does not have the necessary rights. However, in some setups it can be permissible to cache authenticated documents.

An example cache section might look as follows:

The list of cachable documents determines which documents are cached, though the Dispatcher never caches a document in the following circumstances:

• If the HTTP method is not GET.
  Other common methods are POST for form data and HEAD for the HTTP header.
• If the request URI contains a question mark ("?").
  This usually indicates a dynamic page, such as a search result that does not need to be cached.
• The file extension is missing.
  The web server needs the extension to determine the document type (the MIME-type).
• The authentication header is set (this can be configured)

If you do not have dynamic pages (beyond those already excluded by the above rules), you can let the Dispatcher cache everything. The rules section for this looks as follows:

If there are some sections of your page that are dynamic (for example a news application) or within a closed user group, you can define exceptions:

Compression (Apache 1.3 only)

On Apache 1.3 web servers you can also compress the cached documents. This allows Apache to return the document in a compressed form if so requested by the client.

The following rule caches all documents in compressed form; Apache can return either the uncompressed or the compressed form to the client:

This defines a list of all documents that are automatically rendered invalid after a content update.

With auto-invalidation the Dispatcher does not physically delete the pages after a content update, but checks them for validity when they are next requested. Documents in the cache that are not auto-invalidated will remain in the cache until they are deleted by a content update.

Auto-invalidate is typically used for HTML pages. As these often contain navigation elements and links to other pages, it is very hard to determine whether a page is affected by a content update. To stay on the safe side, you usually auto-invalidate all HTML pages. An example configuration for this looks as follows:

With this configuration, when /content/geometrixx/en.html is replicated:

• all the files with pattern en.* are removed from the /content/geometrixx/ folder
• the /content/geometrixx/en/jcr_content folder is removed
• all the other files that match the /invalidate configuration are not deleted: they will be deleted with the next request. In our example /content/geometrixx.html is not deleted, it will be deleted when /content/geometrixx.html is requested.

If you offer automatically generated PDF and ZIP files for download, you might have to auto-invalidate these as well. A configuration example this looks as follows:

An ACL defines specific clients that are allowed to flush the cache. The globbing patterns are matched against the IP.

The following example:

1. denies access to any client
2. explicitly allows access to the localhost

The /statistics section defines the categories that the Dispatcher uses for load balancing estimates.

The Dispatcher keeps a list of typical response times per CQ instance and per request category. When the Dispatcher needs to render a document, it:

• checks these lists to determine which CQ instance currently has the most resources available to handle it
• reserves these resources when considering future requests.

The statistics are internal and cannot be accessed from outside. By default, the Dispatcher makes a distinction between HTML documents and everything else. The default /statistics section looks as follows:

You can define one folder that contains sticky documents; this will be accessed using the URL.

The Dispatcher sends all requests, from a single user, that are in this folder to the same web server. This ensures that session data is present and consistent for all documents. This mechanism uses the renderid cookie. To define sticky connections (which will also set the cookie as necessary) for the folder /myFolder, use the following line:

When a 500 status code occurs the specified "health check" page is checked. If this page also returns a 500 status code the instance is considered to be unavailable and a configurable time penalty is applied before retrying.

retryDelay is the intermediate sleep time (defined in seconds) that is applied after one render was unresponsive and before contacting the next one in the list of "renders".

"1" is the default value used if not explicitly defined. The default should be applicable in most case, so this setting should hardly ever need changing.

numberOfRetries is the number of times that the dispatcher should try to reach any render from the farm before returning a failure. This number could be greater than the number of renders.

"5" is the default value used if not explicitly defined.

unavailablePenalty is the number of seconds that is applied to the Dispatcher statistics after a render is unresponsive. In the Dispatcher statistics the total response time for this render is then increased by the appropriate number of seconds, which in turn moves it to the end of the list of renders as they are sorted by increasing response time.

This might happen, for example, when the TCP/IP connection to the designated hostname/port cannot be established, either because CQ5 is not running (and not listening) or because of a network-related problem.

In the web server configuration, you can set:

• The location of the Dispatcher log file.
  
• The log level.

Refer to the web server documentation and the readme file of your Dispatcher instance for more information.

Apache Rotated / Piped Logs

If using an Apache web server you can use the standard functionality for rotated and/or piped logs. For example, using piped logs:

    DispatcherLog "| /usr/apache/bin/rotatelogs logs/dispatcher.log%Y%m%d 604800"

This will automatically rotate:

• the dispatcher log file; with a timestamp in the extension (logs/dispatcher.log%Y%m%d).
  
• on a weekly basis (60 x 60 x 24 x 7 = 604800 seconds).
  

Please see the Apache web server documentation on Log Rotation and Piped Logs; for example Apache 2.2.

To confirm basic operation and interaction of the web server, dispatcher and CQ instance you can use the following steps:

1. Set the loglevel to 3.
2. Start the web server; this also starts the Dispatcher.
3. Start the CQ instance.
4. Check the log and error files for your web server and the Dispatcher.
   Depending on your web server you should see messages such as:
          [Thu May 30 05:16:36 2002] [notice] Apache/2.0.50 (Unix) configured
   and:
          [Fri Jan 19 17:22:16 2001] [I] [19096] Dispatcher initialized (build XXXX)
5. Surf the website via the web server. Confirm that content is being shown as required.
   For example, on a local installation where CQ runs on port 4502 and the web server on 80 access the Websites console using both:
       http://localhost:4502/libs/wcm/core/content/siteadmin.html
       http://localhost:80/libs/wcm/core/content/siteadmin.html
   The results should be identical. Confirm access to other pages with the same mechanism.
6. Check the contents of the cache directory - is it being filled?
7. Activate a page - check that the cache is being flushed correctly.
8. If everything is operating correctly you can reduce the loglevel to 0 again.
   

If the Dispatcher is being used with CQ the interaction must be configured to ensure clean cache management.

Dependent on your environment, the configuration selected can also increase performance.

A replication agent is used to send a cache invalidation request from the CQ author environment to the Dispatcher when a page is published; this removes the old page content from the cache when new content is published.

To set up a CQ authoring environment, so that it invalidates the cache upon publication of a page:

1. Open the CQ Tools console.
2. Open the required replication agent; for example the Dispatcher Flush agent that is included in a standard installation.
3. In the Settings tab ensure that Enabled is active.
   
4. Open the Transport tab and enter the URI needed to access the dispatcher.
   If you are using the standard Dispatcher Flush agent you will probably need to update the hostname and port; for example, http://<dispatcherHost>:<portApache>/dispatcher/invalidate.cache
   
5. Configure other parameters as required.Configure other parameters as required.
   
6. Click OK to activate the agent.

• The Dispatcher must be reachable from the authoring instance. If your network (e.g. the firewall) is configured such that access between the two is restricted, this may not be the case.

• Publication and cache invalidation take place at the same time. Depending on the timing, a user may request a page just after it was removed from the cache, and just before the new page is published. CQ now returns the old page, and the Dispatcher caches it again. This is more of an issue for large sites.

Various default accounts are included within the CQ installation. Those related to the Dispatcher, must be either changed, configured or deleted as appropriate.

For more information see Default Users and Groups.

Under certain circumstances performance gains can be made by transferring cache management from the authoring environment to a publishing instance. It will then be the publishing environment (not the CQ authoring environment) that sends a cache invalidation request to the Dispatcher when a published page is received.

Such circumstances include:

• preventive action to avoid possible timing conflicts between the Dispatcher and the publish environment (see Invalidating Dispatcher cache from the Authoring Environment).
• several publishing environments residing on high performance servers, but only one authoring environment.

The dispatcher flush is controlled by a replication agent operating on the publish instance. However, the configuration is made on the authoring environment and then transferred by activating the agent:

1. Open the CQ Tools console.
2. Open the required replication agent; for example the Dispatcher Flush agent under Agents on Publish that is included in a standard installation.
3. In the Settings tab ensure that Enabled is active.
   
4. Open the Transport tab and enter the URI needed to access the dispatcher.
   If you are using the standard Dispatcher Flush agent you will probably need to update the hostname and port; for example, http://<dispatcherHost>:<portApache>/dispatcher/invalidate.cache
   
5. Configure other parameters as required.Configure other parameters as required.
6. Repeat for every publish instance affected.
7. If you now activate a page from author to publish, you can see that this agent will initiate a standard replication. The log will include entries indicating requests coming from your publish server; for example:

<publishserver> 13:29:47 127.0.0.1 POST /dispatcher/invalidate.cache 200

In complex setups, you may use multiple Dispatchers. For example, you may use:

• one Dispatcher to publish a website on the Intranet
• a second Dispatcher, under a different address and with different security settings, to publish the same content on the Internet.

In such a case, make sure that each request goes through only one Dispatcher. A Dispatcher does not handle requests that come from another Dispatcher. Therefore, make sure that both Dispatchers access the CQ website directly.

You can use the Dispatcher to seal off sensitive areas. If a dispatcher is in front of a publish instance, you can define a filter that refuses all requests to specified sensitive areas. Requests to the sensitive area then result in a 404 error code (page not found).

See how to define filters in the dispatcher.any file.

As always the first steps are to check the basics:

• Confirm Basic Operation
• Check all log files for your web server and dispatcher. If necessary increase the loglevel used for the dispatcher logging.
• Check your configuration:
  • Do you have multiple Dispatchers? 
    • Have you determined which Dispatcher is handling the website / page you are investigating?
  • Have you implemented filters?
    
    • Are these impacting the matter you are investigating?

IIS provides various trace tools, dependent on the actual version:

• IIS 6 - IIS diagnostic tools can be downloaded and configured
  
• IIS 7 - tracing is fully integrated

These can help you monitor activity.

When using IIS you might experience 404 Not Found being returned in various scenarios. If so, see the following Knowledge Base articles.

• IIS 6/7: POST method returns 404
• IIS 6: Requests to URLs that contain the base path /bin return 404 Not Found

You should also check that the dispatcher cache root and the IIS document root are set to the same directory.

Symptoms

Problems trying to delete workflow models when accessing a CQ author instance through the Dispatcher.

Steps to reproduce:

1. Log in to your CQ author instance (confirm that requests are being routed through the dispatcher).
2. Create a new workflow; for example, with the Title set to workflowToDelete.
3. Confirm that the workflow was successfully created.
4. Select and right click on the workflow, then click Delete.
   
5. Click Yes to confirm.
6. An error message box will appear showing:
         "ERROR 'Could not delete workflow model!!".

Resolution

Add the following headers to the /clientheaders section of your dispatcher.any file:

• x-http-method-override
• x-requested-with

{
  {
  /clientheaders
    {
    ...
    "x-http-method-override"
    "x-requested-with"
    }

This describes how the dispatcher interacts with mod_dir inside the Apache webserver, as this can lead to various, potentially unexpected effects:

In Apache 1.3 mod_dir handles every request where the URL maps to a directory in the file system.

It will either:

• redirect the request to an existing index.html file 
• generate a directory listing

When the dispatcher is enabled, it processes such requests by registering itself as a handler for the content type httpd/unix-directory.

In Apache 2.x things are different. A module can handle different stages of the request, such as URL fixup. mod_dir handles this stage by redirecting a request (when the URL maps to a directory) to the URL with a / appended.

The dispatcher does not intercept the mod_dir fixup, but it will completely handle the request to the redirected URL (i.e. with / appended). This might pose a problem if the remote server (e.g. CQ5) handles requests to /a_path differently to requests to /a_path/ (when /a_path maps to an existing directory).

If this happens you must either:

• disable mod_dir for the Directory or Location subtree handled by the dispatcher
  
• use DirectorySlash Off to configure mod_dir not to append /