June 19, 2025

LiveSite Content Services Performance Issue

We have discovered a serious performance issue in the OpenText LiveSite Content Services product, caused by a change that was introduced in major version 22.2.0.0 and patch release version 21.4.4.0.

The change that was introduced causes every request made to the LiveSite Content Services application both in OpenText Web (TeamSite) and in Runtime environments to create a new connection to ZooKeeper. As shown in the screenshot below, there were over 2000 open connections on one of the customer’s servers.

Open ZooKeeper Connections

Previously, a single connection was open and shared by all requests, which relies on internal load balancing and scheduling within ZooKeeper and Solr to route requests most efficiently.

Code Change Causing Problem

We have traced the problem to the following code found in the com.interwoven.solrutils.impl.SolrServiceHelper.class which is located in the solrutilsimpl.jar file:

Previous Code:

private synchronized CloudSolrClient getCloudSolrClient() {
    CloudSolrClient.Builder builder = (CloudSolrClient.Builder)builders.get(this.mServerDetails.getZookeeperUrl());
    if (builder == null) {
        builder = (new CloudSolrClient.Builder()).withZkHost(this.mServerDetails.getZookeeperUrl());
        builders.put(this.mServerDetails.getZookeeperUrl(), builder);
    }

    return builder.build();
}

Updated Code (notice the usage of the ‘new’ keyword):

private CloudSolrClient getCloudSolrClient() {
    CloudSolrClient.Builder builder = new CloudSolrClient.Builder(
        Collections.singletonList(this.mServerDetails.getZookeeperUrl()), 
        Optional.empty());

    return builder.build();
}

Negative Performance Impacts

Performance is negatively impacted in the following ways:

  1. Each LiveSite Content Services call takes longer to execute as it needs to create a new connectin object. This leads to slower web site component rendering.

  2. All of these open connection objects eventually lead to the Apache Tomcat application server running out of available direct access memory into the JVM. Once memory is exhausted, the following exception is thrown:

     zookeeper.ClientCnxn$SendThread (ClientCnxn.java:1285) - Session 0x100c52b372e40ee for server localhost/[0:0:0:0:0:0:0:1]:2181, Closing socket connection. Attempting reconnect except it is a SessionExpiredException.
 java.lang.OutOfMemoryError: Cannot reserve 952 bytes of direct buffer memory (allocated: 268435381, limit: 268435456)
     at java.nio.Bits.reserveMemory(Bits.java:178) ~[?:?]
     at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:111) ~[?:?]
     at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:360) ~[?:?]
     at sun.nio.ch.Util.getTemporaryDirectBuffer(Util.java:242) ~[?:?]
     at sun.nio.ch.IOUtil.read(IOUtil.java:303) ~[?:?]
     at sun.nio.ch.IOUtil.read(IOUtil.java:269) ~[?:?]
     at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:425) ~[?:?]
     at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:74) ~[zookeeper-3.9.2.jar:3.9.2]
     at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) ~[zookeeper-3.9.2.jar:3.9.2]
     at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1274) ~[zookeeper-3.9.2.jar:3.9.2]

  3. OpenText Web (TeamSite) and LiveSite Content Services applicatoin servers will become unresponsive, slow and unstable. This will lead to restarts being required for the applications to continue to function properly and clear out the exhausted memory and hung ZooKeeper connections.

Solution

The solrutilsimpl.jar file is found in many places within the OpenText Web (TeamSite), Indexer, LiveSite Content Services authoring and runtime applications. This class will need to be updated in all places.

We have reported the issue and await a permanent fix from OpenText.

If you are running any OpenText Web (TeamSite) version 21.4.4.0 or newer, we can provide you the steps for the temporary fix.

Get the Details