ClusteredCache (com.bowman.cardserv.ClusteredCache)
---------------------------------------------------

Cache-sharing with the csproxy is a powerful tool that can be used in a number of different ways. Before contemplating
the various possible scenarios, the basic cache model needs to be understood.

Whenever a proxy node receives a new ecm request from a client, it will query the cache to see if this is the first
time this particular request has been received. One of three things can happen:

1. The ecm has been received before, and a cw reply has already been fetched from a card. This causes an instant hit.
2. The ecm has been received before, but the card transaction is still ongoing and the reply not yet available.
   This causes the request to be held in the cache for a maximum of 'max-cache-wait' seconds. As soon as a reply is
   available all threads waiting for it will be released with their own copy of the cw.
3. The ecm has not been seen before, no card transaction is pending for it. This will place the ecm in a list of pending
   requests, and it will be up to the proxy service mapper and connection manger to find a card to handle it.

The cache knows nothing about which ca-system the ecm belongs to, what sid it is for, or who is asking for it. It will
simply check its lists of already handled ecm -> cw pairs, and the list of pending ecm requests and see which (if any)
contains the new ecm. If two different profiles, or multiple services within one profile, happen to use the same ecm's
the cache will score hits. The chances of a false positive are insignificant enough to be safely ignored.

When using remote cache-sharing, all of the above remains pretty much the same. However, whenever something is added
to the list of pending requests or requests with cw replies, the cache will broadcast the new addition to the remote
proxy (or proxies) that it has been configured to talk to. This broadcast can be done in several ways, but all involve
udp communication and proprietary java object serialization.

Note that in any cache cluster, there can still be occasions when two or more proxies will attempt to query a card for
the same ecm. This will occur when the ecms arrive at both proxies at exactly the same time (which is of course often
the case). Depending on the roundtrip ping time between the proxies, this can be enough to reduce the effectiveness
of the cache significantly. Typically though, you have multiple proxies for the sake of redundancy, you probably want
the same ecm processed in several places so that if one proxy fails to produce a reply, another may succeed.

That said, it is possible to achieve a strict synchronization between cache-instances by using the experimental 
arbitration feature, which introduces a negotiation procedure for each ecm, to determine which proxy is best suited to
handle it (and then only that proxy will proceed with forwarding, the others will wait). See sync-period below.

------------------------------------------------------

The following are settings are available, see proxy.xml for separate examples:

<cache-handler class="com.bowman.cardserv.ClusteredCache">

- To use the ClusteredCache as the cache-handler for the proxy, use the class name: com.bowman.cardserv.ClusteredCache
Changing cache-handlers requires a proxy restart.

<cache-config>
  <cw-max-age>19</cw-max-age>
  <max-cache-wait>7</max-cache-wait>
      
- Inherited from DefaultCache (see proxy-reference.html).

  <remote-host>peer.proxy.host.com</remote-host>
  <remote-port>54278</remote-port>

- One way of specifying where to send cache-updates, when there is only a single target proxy.

  <multicast-group>230.2.3.2</multicast-group>
  <multicast-ttl>2</multicast-ttl>
  <remote-port>54278</remote-port>

- Another way to configure targets for cache-updates. Multicast typically only works in a LAN environment.

  <tracker-url>http://cstracker.host.com/list.enc</tracker-url>
  <tracker-key>secretkey</tracker-key>
  <tracker-update>10</tracker-update> <!-- minutes -->

- A third (and perhaps the best) option for configuring multiple targets for cache-updates. The ClusteredCache will
fetch a plain text list of hostnames and portnumbers, and send updates to every entry in the list. The list can be
automatically fetched at regular intervals (or if tracker-update is 0, only when proxy.xml is modified/touched).
This approach allows proxies to be added to a cluster without having to modify the configuration of already existing nodes.
The list must have the following format (# are comments):

# ClusteredCache list file. Syntax: hostname_or_ip:udp_port_nr
proxy1.host.com:54278
proxy2.host.com:54278
192.168.0.3:54275

The list can be stored anywhere, as long as it can be accessed via url (e.g file://, http, https, ftp). 
Optionally, the list can also be blowfish encrypted using the included tool trtest.jar (found in lib, java -jar trtest.jar).
If encrypted, tracker-key must be correctly set.
 
  <local-port>54278</local-port>  
  
- The UDP port where this ClusteredCache instance will listen for incoming updates.  
     
  <local-host>csproxy3.host.com</local-host>
  
- The external hostname or IP of this ClusteredCache. This is only needed when using the above tracker setup. The name
should match the one in the tracker list, so the cache instance can identify itself in the list and avoid sending itself
updates.

  <debug>true</debug>

- Set debug to true to enable additional cache information in the status-web. This can impact performance, use with care.

  <hide-names>false</hide-names>

- If configured to send to one or more remote caches, this controls whether the names of the connectors are censored in
the outgoing cache updates (will appear as remote: unknown to the target). Only makes sense when dealing with untrusted
proxy peers.

  <sync-period>200</sync-period> <!-- milliseconds -->
  
- Set this > 0 to enable the experimental arbitration procedure. For this example, the ClusteredCache would use 200 ms
for every new (previously unseen ecm) to wait and synchronize with as many other proxies in the cluster as possible, and
determine who is best suited to handle it. Only this proxy would proceed with a forward to a card. This adds 200 ms to 
every single transaction, but should ensure that a cluster of proxies will only ask one card in one proxy, once, for the 
same ecm. Probably only usable in a cluster where all nodes are fully trusted, and where the network is reliable with 
fairly fixed ping times and no congestion.

</cache-config>

------------------------------------------------------

NOTE: It is possible to set up the ClusteredCache without any remote targets (receive-only-mode). If no remote-host/port
is set using any of the various methods then the default behaviour will be to not attempt any sending of updates.
This is useful when creating a cache-only proxy node, receiving updates from multiple other proxies but sending to none.
It can also be used to augment local cards with additional services (which will only be available through the cache).

