3437588 - Solr full indexing prevents the CCv2 pods from starting up successfully

The pods with Backoffice webapp deployed (usually backoffice pods) are trying to perform solr full indexing during startup and are getting restarted before are able to start.

Following logs may appear on affected pods aiming the indexing was invoked during Backoffice initializing:

{"instant":{"epochSecond":1708933608,"nanoOfSecond":546137864},"thread":"localhost-startStop-1","level":"INFO","loggerName":"com.hybris.backoffice.solrsearch.setup.BackofficeSolrSearchIndexInitializer","message":"Index for 'Solr Config for Backoffice' configuration not initialized","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","contextMap":{"Tenant":""},"threadId":24,"threadPriority":5,"origin":"PLATFORM"}

{"instant":{"epochSecond":1708933608,"nanoOfSecond":546270365},"thread":"localhost-startStop-1","level":"INFO","loggerName":"com.hybris.backoffice.solrsearch.setup.BackofficeSolrSearchIndexInitializer","message":"Performing FULL INDEX operation for 'Solr Config for Backoffice' configuration","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","contextMap":{"Tenant":""},"threadId":24,"threadPriority":5,"origin":"PLATFORM"}

The thread dump contains following thread:

"localhost-startStop-1" #24 daemon prio=5 os_prio=0 cpu=145945.07ms elapsed=381.05s allocated=46725M defined_classes=39780 tid=0x00007f210001e160 nid=0x1b3 waiting on condition  [0x00007f2123175000]
   java.lang.Thread.State: WAITING (parking)
    at jdk.internal.misc.Unsafe.park(java.base@17.0.10/Native Method)
    - parking to wait for  <0x000000062be000b8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.park(java.base@17.0.10/LockSupport.java:341)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(java.base@17.0.10/AbstractQueuedSynchronizer.java:506)
    at java.util.concurrent.ForkJoinPool.unmanagedBlock(java.base@17.0.10/ForkJoinPool.java:3465)
    at java.util.concurrent.ForkJoinPool.managedBlock(java.base@17.0.10/ForkJoinPool.java:3436)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@17.0.10/AbstractQueuedSynchronizer.java:1623)
    at java.util.concurrent.LinkedBlockingQueue.take(java.base@17.0.10/LinkedBlockingQueue.java:435)
    at java.util.concurrent.ExecutorCompletionService.take(java.base@17.0.10/ExecutorCompletionService.java:200)
    at de.hybris.platform.solrfacetsearch.indexer.strategies.impl.DefaultIndexerStrategy.runWorkers(DefaultIndexerStrategy.java:131)
    at de.hybris.platform.solrfacetsearch.indexer.strategies.impl.DefaultIndexerStrategy.doExecute(DefaultIndexerStrategy.java:88)
    at de.hybris.platform.solrfacetsearch.indexer.strategies.impl.AbstractIndexerStrategy.doExecute(AbstractIndexerStrategy.java:160)
    at com.hybris.backoffice.solrsearch.indexer.impl.BackofficeIndexerStrategy.doExecuteAsAdmin(BackofficeIndexerStrategy.java:55)
    at com.hybris.backoffice.solrsearch.indexer.impl.BackofficeIndexerStrategy$1.executeWithoutResult(BackofficeIndexerStrategy.java:38)
    at de.hybris.platform.servicelayer.session.SessionExecutionBody.execute(SessionExecutionBody.java:21)
    at de.hybris.platform.servicelayer.session.impl.DefaultSessionService.executeInLocalView(DefaultSessionService.java:124)
    at com.hybris.backoffice.solrsearch.indexer.impl.BackofficeIndexerStrategy.doExecute(BackofficeIndexerStrategy.java:31)
    at de.hybris.platform.solrfacetsearch.indexer.strategies.impl.AbstractIndexerStrategy.execute(AbstractIndexerStrategy.java:119)
    at de.hybris.platform.solrfacetsearch.indexer.impl.DefaultIndexerService.performFullIndex(DefaultIndexerService.java:61)
    at de.hybris.platform.solrfacetsearch.indexer.impl.DefaultIndexerService.performFullIndex(DefaultIndexerService.java:46)
    at com.hybris.backoffice.solrsearch.setup.BackofficeSolrSearchIndexInitializer.initializeIndex(BackofficeSolrSearchIndexInitializer.java:83)
    at com.hybris.backoffice.solrsearch.setup.BackofficeSolrSearchIndexInitializer$$Lambda$2369/0x00000008021d47b8.accept(Unknown Source)
    at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(java.base@17.0.10/ForEachOps.java:183)
    at java.util.stream.ReferencePipeline$2$1.accept(java.base@17.0.10/ReferencePipeline.java:179)
    at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(java.base@17.0.10/ArrayList.java:1625)
    at java.util.stream.AbstractPipeline.copyInto(java.base@17.0.10/AbstractPipeline.java:509)
    at java.util.stream.AbstractPipeline.wrapAndCopyInto(java.base@17.0.10/AbstractPipeline.java:499)
    at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(java.base@17.0.10/ForEachOps.java:150)
    at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(java.base@17.0.10/ForEachOps.java:173)
    at java.util.stream.AbstractPipeline.evaluate(java.base@17.0.10/AbstractPipeline.java:234)
    at java.util.stream.ReferencePipeline.forEach(java.base@17.0.10/ReferencePipeline.java:596)
    at com.hybris.backoffice.solrsearch.setup.BackofficeSolrSearchIndexInitializer.initializeIndexesIfNecessary(BackofficeSolrSearchIndexInitializer.java:37)
    at com.hybris.backoffice.search.setup.AbstractBackofficeSearchIndexInitializer.initialize(AbstractBackofficeSearchIndexInitializer.java:15)

SAP Commerce Cloud

KBA , CEC-SCC-CLA-DEP , Deployments , Problem