Symptom
The data migration process remains stuck for several hours without making progress. No explicit errors are reported in the logs, and there is no indication of database resource exhaustion.
In platform-update pod log, below message can be seen, and almost no new logs being printed only with few audit log.
| {"instant":{"epochSecond":1773104687,"nanoOfSecond":133020357},"thread":"main","level":"INFO","loggerName":"de.hybris.platform.cronjob.jalo.SessionCtxValueJDBCCleaner","message":"########################################################################################","contextMap":{"CronJob":"","Tenant":""},"endOfBatch":false,"loggerFqcn":"org.apache.log4j.Category","threadId":1,"threadPriority":5,"origin":"PLATFORM"} {"instant":{"epochSecond":1773104687,"nanoOfSecond":133076211},"thread":"main","level":"INFO","loggerName":"de.hybris.platform.cronjob.jalo.SessionCtxValueJDBCCleaner","message":"Performing SessionCtxValueJDBCCleaner.\nIf it is not desired, switch the jdbc.clean.sessionContextValues flag to false (or remove it completely)","contextMap":{"CronJob":"","Tenant":""},"endOfBatch":false,"loggerFqcn":"org.apache.log4j.Category","threadId":1,"threadPriority":5,"origin":"PLATFORM"} {"instant":{"epochSecond":1773104687,"nanoOfSecond":133102040},"thread":"main","level":"INFO","loggerName":"de.hybris.platform.cronjob.jalo.SessionCtxValueJDBCCleaner","message":"########################################################################################","contextMap":{"CronJob":"","Tenant":""},"endOfBatch":false,"loggerFqcn":"org.apache.log4j.Category","threadId":1,"threadPriority":5,"origin":"PLATFORM"} |
After capturing a thread dump from the migration pod, the following stack trace is observed on the main thread:
| at de.hybris.platform.servicelayer.internal.model.impl.DefaultModelService.saveAllInternal(DefaultModelService.java:702) at de.hybris.platform.servicelayer.internal.model.impl.DefaultModelService.saveAll(DefaultModelService.java:682) at de.hybris.platform.servicelayer.internal.model.impl.DefaultModelService.save(DefaultModelService.java:629) at de.hybris.platform.cloud.storage.service.impl.DefaultMediaDeletionService.trackDeletion(DefaultMediaDeletionService.java:40) at de.hybris.platform.cloud.storage.CloudAzureBlobStorageStrategy.delete(CloudAzureBlobStorageStrategy.java:178) at de.hybris.platform.jalo.media.MediaManager.deleteMedia(MediaManager.java:287) at de.hybris.platform.jalo.media.MediaManager.deleteMediaDataUnlessReferenced(MediaManager.java:332) at de.hybris.platform.media.interceptors.RemoveDataOnMediaModelRemoveInterceptor.onRemove(RemoveDataOnMediaModelRemoveInterceptor.java:30) at de.hybris.platform.media.interceptors.RemoveDataOnMediaModelRemoveInterceptor.onRemove(RemoveDataOnMediaModelRemoveInterceptor.java:17) at de.hybris.platform.servicelayer.internal.model.impl.wrapper.ModelWrapper.invokeRemoveInterceptors(ModelWrapper.java:331) at de.hybris.platform.servicelayer.internal.model.impl.wrapper.ModelWrapper.executeInterceptorsAndCascade(ModelWrapper.java:108) at de.hybris.platform.servicelayer.internal.model.impl.wrapper.ModelWrapper.executeInterceptorsAndCascade(ModelWrapper.java:127) at de.hybris.platform.servicelayer.internal.model.extractor.impl.DefaultModelExtractor.process(DefaultModelExtractor.java:35) at de.hybris.platform.servicelayer.internal.model.impl.DefaultModelService.performPersistenceOperations(DefaultModelService.java:707) at de.hybris.platform.servicelayer.internal.model.impl.DefaultModelService.removeAllInternal(DefaultModelService.java:590) at de.hybris.platform.servicelayer.internal.model.impl.DefaultModelService.removeAll(DefaultModelService.java:578) at de.hybris.platform.servicelayer.internal.model.impl.DefaultModelService.removeAll(DefaultModelService.java:531) at de.hybris.platform.servicelayer.internal.model.impl.DefaultModelService.remove(DefaultModelService.java:511) at de.hybris.platform.task.impl.LegacyOAuthCronJobsCleanUp$$Lambda/0x0000000801fc2f78.accept(Unknown Source) at java.util.ArrayList.forEach(java.base@21.0.9/ArrayList.java:1596) at java.util.Collections$UnmodifiableCollection.forEach(java.base@21.0.9/Collections.java:1117) at de.hybris.platform.task.impl.LegacyOAuthCronJobsCleanUp.removeLegacyOAuthCleaningCronJobs(LegacyOAuthCronJobsCleanUp.java:54) at java.lang.invoke.LambdaForm$DMH/0x00000008003bd400.invokeVirtual(java.base@21.0.9/LambdaForm$DMH) at java.lang.invoke.LambdaForm$MH/0x00000008007cb000.invoke(java.base@21.0.9/LambdaForm$MH) |
Stuck issue still persists after resolution has been applied, here is the stack trace of main thread, a bit difference compare to the above trace.
java.lang.Thread.State: RUNNABLE at java.util.AbstractCollection.addAll(java.base@21.0.11/AbstractCollection.java:338) at java.util.LinkedHashSet.<init>(java.base@21.0.11/LinkedHashSet.java:182) at de.hybris.platform.servicelayer.internal.model.impl.wrapper.ModelWrapper.filterElements(ModelWrapper.java:190) at de.hybris.platform.servicelayer.internal.model.impl.wrapper.ModelWrapper.getModelsToDelete(ModelWrapper.java:179) at de.hybris.platform.servicelayer.internal.model.impl.wrapper.ModelWrapper.executeInterceptorsAndCascade(ModelWrapper.java:117) at de.hybris.platform.servicelayer.internal.model.impl.wrapper.ModelWrapper.executeInterceptorsAndCascade(ModelWrapper.java:127) at de.hybris.platform.servicelayer.internal.model.extractor.impl.DefaultModelExtractor.process(DefaultModelExtractor.java:35) at de.hybris.platform.servicelayer.internal.model.impl.DefaultModelService.performPersistenceOperations(DefaultModelService.java:707) at de.hybris.platform.servicelayer.internal.model.impl.DefaultModelService.removeAllInternal(DefaultModelService.java:590) at de.hybris.platform.servicelayer.internal.model.impl.DefaultModelService.removeAll(DefaultModelService.java:578) at de.hybris.platform.servicelayer.internal.model.impl.DefaultModelService.removeAll(DefaultModelService.java:531) at de.hybris.platform.servicelayer.internal.model.impl.DefaultModelService.remove(DefaultModelService.java:511) at de.hybris.platform.task.impl.LegacyOAuthCronJobsCleanUp$$Lambda/0x000000080205b5b8.accept(Unknown Source) at java.util.ArrayList.forEach(java.base@21.0.11/ArrayList.java:1596) at java.util.Collections$UnmodifiableCollection.forEach(java.base@21.0.11/Collections.java:1117) at de.hybris.platform.task.impl.LegacyOAuthCronJobsCleanUp.removeLegacyOAuthCleaningCronJobs(LegacyOAuthCronJobsCleanUp.java:54) at java.lang.invoke.LambdaForm$DMH/0x00000008001c0400.invokeVirtual(java.base@21.0.11/LambdaForm$DMH) at java.lang.invoke.LambdaForm$MH/0x0000000800763000.invoke(java.base@21.0.11/LambdaForm$MH) at java.lang.invoke.Invokers$Holder.invokeExact_MT(java.base@21.0.11/Invokers$Holder) at jdk.internal.reflect.DirectMethodHandleAccessor.invokeImpl(java.base@21.0.11/DirectMethodHandleAccessor.java:153) at jdk.internal.reflect.DirectMethodHandleAccessor.invoke(java.base@21.0.11/DirectMethodHandleAccessor.java:103) at java.lang.reflect.Method.invoke(java.base@21.0.11/Method.java:580) at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:281) at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:265) at de.hybris.platform.core.initialization.impl.SystemSetupCollectorImpl.invokeSystemSetupMethod(SystemSetupCollectorImpl.java:322) at de.hybris.platform.core.initialization.impl.SystemSetupCollectorImpl.executeMethods(SystemSetupCollectorImpl.java:278) at de.hybris.platform.core.Initialization.createEssentialData(Initialization.java:1139) at de.hybris.platform.core.Initialization.doInitializeImpl(Initialization.java:657) at de.hybris.platform.core.Initialization$5.call(Initialization.java:935) at de.hybris.platform.core.Initialization$5.call(Initialization.java:926) at de.hybris.platform.core.system.InitializationLockHandler.performLocked(InitializationLockHandler.java:70) at de.hybris.platform.core.Initialization.doInitialize(Initialization.java:967) at de.hybris.ant.taskdefs.InitPlatformAntPerformableImpl.performImpl(InitPlatformAntPerformableImpl.java:106) at de.hybris.ant.taskdefs.AbstractAntPerformable.doPerform(AbstractAntPerformable.java:84) at java.lang.invoke.DirectMethodHandle$Holder.invokeSpecial(java.base@21.0.11/DirectMethodHandle$Holder) at java.lang.invoke.LambdaForm$MH/0x0000000800763000.invoke(java.base@21.0.11/LambdaForm$MH) at java.lang.invoke.Invokers$Holder.invokeExact_MT(java.base@21.0.11/Invokers$Holder) at jdk.internal.reflect.DirectMethodHandleAccessor.invokeImpl(java.base@21.0.11/DirectMethodHandleAccessor.java:153) at jdk.internal.reflect.DirectMethodHandleAccessor.invoke(java.base@21.0.11/DirectMethodHandleAccessor.java:103) at java.lang.reflect.Method.invoke(java.base@21.0.11/Method.java:580) at bsh.Reflect.invokeMethod(Reflect.java:131) at bsh.Reflect.invokeObjectMethod(Reflect.java:77) at bsh.Name.invokeMethod(Name.java:852) at bsh.BSHMethodInvocation.eval(BSHMethodInvocation.java:69) at bsh.BSHPrimaryExpression.eval(BSHPrimaryExpression.java:96) at bsh.BSHPrimaryExpression.eval(BSHPrimaryExpression.java:41) at bsh.BSHBlock.evalBlock(BSHBlock.java:125) at bsh.BSHBlock.eval(BSHBlock.java:75) at bsh.BSHBlock.eval(BSHBlock.java:41) at bsh.BSHTryStatement.eval(BSHTryStatement.java:80) at bsh.Interpreter.eval(Interpreter.java:659) at bsh.Interpreter.eval(Interpreter.java:750) at bsh.Interpreter.eval(Interpreter.java:739) at de.hybris.platform.util.ClientExecuter.execute(ClientExecuter.java:42) at java.lang.invoke.LambdaForm$DMH/0x000000080033d800.invokeStatic(java.base@21.0.11/LambdaForm$DMH) at java.lang.invoke.LambdaForm$MH/0x000000080033fc00.invoke(java.base@21.0.11/LambdaForm$MH) at java.lang.invoke.LambdaForm$MH/0x000000080014fc00.invokeExact_MT(java.base@21.0.11/LambdaForm$MH) at jdk.internal.reflect.DirectMethodHandleAccessor.invokeImpl(java.base@21.0.11/DirectMethodHandleAccessor.java:156) at jdk.internal.reflect.DirectMethodHandleAccessor.invoke(java.base@21.0.11/DirectMethodHandleAccessor.java:103) at java.lang.reflect.Method.invoke(java.base@21.0.11/Method.java:580) at de.hybris.bootstrap.loader.Loader.execute(Loader.java:144) at de.hybris.bootstrap.loader.Loader.main(Loader.java:108) |
Read more...
Environment
SAP Commerce Cloud for JDK21
Product
Keywords
ccv2, hybris, authentication, trigger, JobPerformable, Migrate Data stuck with JDK 21, Deployment stuck with JDK 21 , KBA , CEC-SCC-CLA-DEP , Deployments , Problem
About this page
This is a preview of a SAP Knowledge Base Article. Click more to access the full version on SAP for Me (Login required).Search for additional results
Visit SAP Support Portal's SAP Notes and KBA Search.
SAP Knowledge Base Article - Preview