01:44:32,729 PM [system1-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:44:33,543 PM [system1-akka.actor.default-dispatcher-6] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-type-1: no initial generation override, starting from 0 01:44:33,545 PM [system1-akka.actor.default-dispatcher-6] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-type-1: persisting new identifier ClientIdentifier{frontend=member-1-frontend-type-1, generation=0} 01:44:33,548 PM [system1-akka.actor.default-dispatcher-6] [DEBUG] AbstractClientActor - member-1-frontend-type-1: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@6b60dba4 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@17a1593a 01:44:33,561 PM [system1-akka.actor.default-dispatcher-7] [DEBUG] SavingClientActorBehavior - member-1-frontend-type-1: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-type-1, 0, 1710164673550, None)) 01:44:33,568 PM [system1-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-type-1: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164673549,0,0)) 01:44:33,573 PM [system1-akka.actor.default-dispatcher-6] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-type-1: recovered identifier ClientIdentifier{frontend=another-frontend-type-2, generation=0} 01:44:33,575 PM [system1-akka.actor.default-dispatcher-6] [ERROR] RecoveringClientActorBehavior - member-1-frontend-type-1: Mismatched frontend identifier, shutting down. Current: member-1-frontend-type-1 Saved: another-frontend-type-2 01:44:33,575 PM [system1-akka.actor.default-dispatcher-6] [DEBUG] AbstractClientActor - member-1-frontend-type-1: shutting down 01:44:33,623 PM [system1-akka.actor.default-dispatcher-6] [INFO] CoordinatedShutdown - Running CoordinatedShutdown with reason [ActorSystemTerminateReason] [INFO] [03/11/2024 13:44:33.617] [main] [CoordinatedShutdown(akka://system1)] Running CoordinatedShutdown with reason [ActorSystemTerminateReason] 01:44:33,662 PM [system1-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-type-1: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@17a1593a to org.opendaylight.controller.cluster.access.client.ClientActorBehavior$MockitoMock$S7168poM@7661e474 [ERROR] [03/11/2024 13:44:33.672] [system1-akka.actor.default-dispatcher-5] [akka://system1/user/$a] Cannot invoke "org.opendaylight.controller.cluster.messaging.MessageAssembler.close()" because ".responseMessageAssembler" is null java.lang.NullPointerException: Cannot invoke "org.opendaylight.controller.cluster.messaging.MessageAssembler.close()" because ".responseMessageAssembler" is null at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.close(ClientActorBehavior.java:117) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.close(ClientActorBehavior.java:116) at org.opendaylight.controller.cluster.access.client.AbstractClientActor.postStop(AbstractClientActor.java:37) at akka.actor.Actor.aroundPostStop(Actor.scala:556) at akka.actor.Actor.aroundPostStop$(Actor.scala:556) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundPostStop(PersistentActor.scala:295) at akka.persistence.Eventsourced.aroundPostStop(Eventsourced.scala:298) at akka.persistence.Eventsourced.aroundPostStop$(Eventsourced.scala:294) at akka.persistence.AbstractPersistentActor.aroundPostStop(PersistentActor.scala:295) at akka.actor.dungeon.FaultHandling.finishTerminate(FaultHandling.scala:242) at akka.actor.dungeon.FaultHandling.terminate(FaultHandling.scala:199) at akka.actor.dungeon.FaultHandling.terminate$(FaultHandling.scala:169) at akka.actor.ActorCell.terminate(ActorCell.scala:410) at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:519) at akka.actor.ActorCell.systemInvoke(ActorCell.scala:535) at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:295) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:273) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:44:33,674 PM [system1-akka.actor.default-dispatcher-6] [ERROR] ActorBehaviorTest$MockedActor - Cannot invoke "org.opendaylight.controller.cluster.messaging.MessageAssembler.close()" because ".responseMessageAssembler" is null java.lang.NullPointerException: Cannot invoke "org.opendaylight.controller.cluster.messaging.MessageAssembler.close()" because ".responseMessageAssembler" is null at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.close(ClientActorBehavior.java:117) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.close(ClientActorBehavior.java:116) at org.opendaylight.controller.cluster.access.client.AbstractClientActor.postStop(AbstractClientActor.java:37) at akka.actor.Actor.aroundPostStop(Actor.scala:556) at akka.actor.Actor.aroundPostStop$(Actor.scala:556) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundPostStop(PersistentActor.scala:295) at akka.persistence.Eventsourced.aroundPostStop(Eventsourced.scala:298) at akka.persistence.Eventsourced.aroundPostStop$(Eventsourced.scala:294) at akka.persistence.AbstractPersistentActor.aroundPostStop(PersistentActor.scala:295) at akka.actor.dungeon.FaultHandling.finishTerminate(FaultHandling.scala:242) at akka.actor.dungeon.FaultHandling.terminate(FaultHandling.scala:199) at akka.actor.dungeon.FaultHandling.terminate$(FaultHandling.scala:169) at akka.actor.ActorCell.terminate(ActorCell.scala:410) at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:519) at akka.actor.ActorCell.systemInvoke(ActorCell.scala:535) at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:295) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:273) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:44:33,745 PM [system1-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:44:33,804 PM [system1-akka.actor.default-dispatcher-8] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-type-1: no initial generation override, starting from 0 01:44:33,804 PM [system1-akka.actor.default-dispatcher-8] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-type-1: persisting new identifier ClientIdentifier{frontend=member-1-frontend-type-1, generation=0} 01:44:33,805 PM [system1-akka.actor.default-dispatcher-8] [DEBUG] AbstractClientActor - member-1-frontend-type-1: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@36cf1d57 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7330b9a 01:44:33,808 PM [system1-akka.actor.default-dispatcher-8] [DEBUG] SavingClientActorBehavior - member-1-frontend-type-1: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-type-1, 0, 1710164673806, None)) [WARN] [03/11/2024 13:44:33.811] [system1-akka.actor.default-dispatcher-8] [akka://system1/user/$a] received dead letter from Actor[akka://system1/system/in-memory-snapshot-store#2035922775]: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164673805,0,0)) [INFO] [akkaDeadLetter][03/11/2024 13:44:33.815] [system1-akka.actor.default-dispatcher-8] [akka://system1/user/$a] Message [akka.persistence.DeleteSnapshotsSuccess] from Actor[akka://system1/system/in-memory-snapshot-store#2035922775] to Actor[akka://system1/user/$a#-786725174] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://system1/user/$a#-786725174] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:44:33,820 PM [system1-akka.actor.default-dispatcher-8] [INFO] RepointableActorRef - Message [akka.persistence.DeleteSnapshotsSuccess] from Actor[akka://system1/system/in-memory-snapshot-store#2035922775] to Actor[akka://system1/user/$a#-786725174] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://system1/user/$a#-786725174] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:44:33,821 PM [system1-akka.actor.default-dispatcher-8] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-type-1: no initial generation override, starting from 0 01:44:33,821 PM [system1-akka.actor.default-dispatcher-8] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-type-1: persisting new identifier ClientIdentifier{frontend=member-1-frontend-type-1, generation=0} 01:44:33,821 PM [system1-akka.actor.default-dispatcher-8] [DEBUG] AbstractClientActor - member-1-frontend-type-1: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@6279d58c to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2eb4b085 01:44:33,822 PM [system1-akka.actor.default-dispatcher-8] [DEBUG] SavingClientActorBehavior - member-1-frontend-type-1: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-type-1, 0, 1710164673822, None)) 01:44:33,826 PM [system1-akka.actor.default-dispatcher-12] [WARN] SavingClientActorBehavior - member-1-frontend-type-1: failed to delete prior snapshots java.lang.RuntimeException: delete failed at org.opendaylight.controller.cluster.access.client.ActorBehaviorTest.testRecoveryAfterRestartDeleteSnapshotsFail(ActorBehaviorTest.java:156) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:728) at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60) at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131) at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156) at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147) at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:86) at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103) at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93) at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37) at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92) at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:218) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:214) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:139) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:69) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141) at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138) at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95) at java.base/java.util.ArrayList.forEach(ArrayList.java:1511) at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141) at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138) at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95) at java.base/java.util.ArrayList.forEach(ArrayList.java:1511) at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141) at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138) at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95) at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35) at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57) at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) 01:44:33,831 PM [system1-akka.actor.default-dispatcher-12] [DEBUG] AbstractClientActor - member-1-frontend-type-1: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2eb4b085 to org.opendaylight.controller.cluster.access.client.ClientActorBehavior$MockitoMock$S7168poM@5c30decf [INFO] [03/11/2024 13:44:33.931] [main] [CoordinatedShutdown(akka://system1)] Running CoordinatedShutdown with reason [ActorSystemTerminateReason] 01:44:33,932 PM [system1-akka.actor.default-dispatcher-12] [INFO] CoordinatedShutdown - Running CoordinatedShutdown with reason [ActorSystemTerminateReason] [ERROR] [03/11/2024 13:44:33.934] [system1-akka.actor.default-dispatcher-6] [akka://system1/user/$b] Cannot invoke "org.opendaylight.controller.cluster.messaging.MessageAssembler.close()" because ".responseMessageAssembler" is null java.lang.NullPointerException: Cannot invoke "org.opendaylight.controller.cluster.messaging.MessageAssembler.close()" because ".responseMessageAssembler" is null at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.close(ClientActorBehavior.java:117) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.close(ClientActorBehavior.java:116) at org.opendaylight.controller.cluster.access.client.AbstractClientActor.postStop(AbstractClientActor.java:37) at akka.actor.Actor.aroundPostStop(Actor.scala:556) at akka.actor.Actor.aroundPostStop$(Actor.scala:556) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundPostStop(PersistentActor.scala:295) at akka.persistence.Eventsourced.aroundPostStop(Eventsourced.scala:298) at akka.persistence.Eventsourced.aroundPostStop$(Eventsourced.scala:294) at akka.persistence.AbstractPersistentActor.aroundPostStop(PersistentActor.scala:295) at akka.actor.dungeon.FaultHandling.finishTerminate(FaultHandling.scala:242) at akka.actor.dungeon.FaultHandling.terminate(FaultHandling.scala:199) at akka.actor.dungeon.FaultHandling.terminate$(FaultHandling.scala:169) at akka.actor.ActorCell.terminate(ActorCell.scala:410) at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:519) at akka.actor.ActorCell.systemInvoke(ActorCell.scala:535) at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:295) at akka.dispatch.Mailbox.run(Mailbox.scala:230) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:44:33,936 PM [system1-akka.actor.default-dispatcher-6] [ERROR] ActorBehaviorTest$MockedActor - Cannot invoke "org.opendaylight.controller.cluster.messaging.MessageAssembler.close()" because ".responseMessageAssembler" is null java.lang.NullPointerException: Cannot invoke "org.opendaylight.controller.cluster.messaging.MessageAssembler.close()" because ".responseMessageAssembler" is null at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.close(ClientActorBehavior.java:117) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.close(ClientActorBehavior.java:116) at org.opendaylight.controller.cluster.access.client.AbstractClientActor.postStop(AbstractClientActor.java:37) at akka.actor.Actor.aroundPostStop(Actor.scala:556) at akka.actor.Actor.aroundPostStop$(Actor.scala:556) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundPostStop(PersistentActor.scala:295) at akka.persistence.Eventsourced.aroundPostStop(Eventsourced.scala:298) at akka.persistence.Eventsourced.aroundPostStop$(Eventsourced.scala:294) at akka.persistence.AbstractPersistentActor.aroundPostStop(PersistentActor.scala:295) at akka.actor.dungeon.FaultHandling.finishTerminate(FaultHandling.scala:242) at akka.actor.dungeon.FaultHandling.terminate(FaultHandling.scala:199) at akka.actor.dungeon.FaultHandling.terminate$(FaultHandling.scala:169) at akka.actor.ActorCell.terminate(ActorCell.scala:410) at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:519) at akka.actor.ActorCell.systemInvoke(ActorCell.scala:535) at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:295) at akka.dispatch.Mailbox.run(Mailbox.scala:230) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:44:33,974 PM [system1-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:44:34,27 PM [system1-akka.actor.default-dispatcher-8] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-type-1: no initial generation override, starting from 0 01:44:34,27 PM [system1-akka.actor.default-dispatcher-8] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-type-1: persisting new identifier ClientIdentifier{frontend=member-1-frontend-type-1, generation=0} 01:44:34,27 PM [system1-akka.actor.default-dispatcher-8] [DEBUG] AbstractClientActor - member-1-frontend-type-1: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@3b93d84a to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@f291e4a 01:44:34,28 PM [system1-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-type-1: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-type-1, 0, 1710164674027, None)) 01:44:34,36 PM [system1-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-type-1: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164674026,0,0)) 01:44:34,38 PM [system1-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-type-1: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@f291e4a to org.opendaylight.controller.cluster.access.client.ClientActorBehavior$MockitoMock$S7168poM@470ad01c [INFO] [03/11/2024 13:44:34.048] [main] [CoordinatedShutdown(akka://system1)] Running CoordinatedShutdown with reason [ActorSystemTerminateReason] 01:44:34,49 PM [system1-akka.actor.default-dispatcher-8] [INFO] CoordinatedShutdown - Running CoordinatedShutdown with reason [ActorSystemTerminateReason] 01:44:34,53 PM [system1-akka.actor.default-dispatcher-8] [ERROR] ActorBehaviorTest$MockedActor - Cannot invoke "org.opendaylight.controller.cluster.messaging.MessageAssembler.close()" because ".responseMessageAssembler" is null java.lang.NullPointerException: Cannot invoke "org.opendaylight.controller.cluster.messaging.MessageAssembler.close()" because ".responseMessageAssembler" is null at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.close(ClientActorBehavior.java:117) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.close(ClientActorBehavior.java:116) at org.opendaylight.controller.cluster.access.client.AbstractClientActor.postStop(AbstractClientActor.java:37) at akka.actor.Actor.aroundPostStop(Actor.scala:556) [ERROR] [03/11/2024 13:44:34.053] [system1-akka.actor.default-dispatcher-8] [akka://system1/user/$a] Cannot invoke "org.opendaylight.controller.cluster.messaging.MessageAssembler.close()" because ".responseMessageAssembler" is null java.lang.NullPointerException: Cannot invoke "org.opendaylight.controller.cluster.messaging.MessageAssembler.close()" because ".responseMessageAssembler" is null at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.close(ClientActorBehavior.java:117) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.close(ClientActorBehavior.java:116) at org.opendaylight.controller.cluster.access.client.AbstractClientActor.postStop(AbstractClientActor.java:37) at akka.actor.Actor.aroundPostStop(Actor.scala:556) at akka.actor.Actor.aroundPostStop$(Actor.scala:556) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundPostStop(PersistentActor.scala:295) at akka.persistence.Eventsourced.aroundPostStop(Eventsourced.scala:298) at akka.persistence.Eventsourced.aroundPostStop$(Eventsourced.scala:294) at akka.persistence.AbstractPersistentActor.aroundPostStop(PersistentActor.scala:295) at akka.actor.dungeon.FaultHandling.finishTerminate(FaultHandling.scala:242) at akka.actor.dungeon.FaultHandling.terminate(FaultHandling.scala:199) at akka.actor.dungeon.FaultHandling.terminate$(FaultHandling.scala:169) at akka.actor.ActorCell.terminate(ActorCell.scala:410) at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:519) at akka.actor.ActorCell.systemInvoke(ActorCell.scala:535) at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:295) at akka.dispatch.Mailbox.run(Mailbox.scala:230) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) at akka.actor.Actor.aroundPostStop$(Actor.scala:556) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundPostStop(PersistentActor.scala:295) at akka.persistence.Eventsourced.aroundPostStop(Eventsourced.scala:298) at akka.persistence.Eventsourced.aroundPostStop$(Eventsourced.scala:294) at akka.persistence.AbstractPersistentActor.aroundPostStop(PersistentActor.scala:295) at akka.actor.dungeon.FaultHandling.finishTerminate(FaultHandling.scala:242) at akka.actor.dungeon.FaultHandling.terminate(FaultHandling.scala:199) at akka.actor.dungeon.FaultHandling.terminate$(FaultHandling.scala:169) at akka.actor.ActorCell.terminate(ActorCell.scala:410) at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:519) at akka.actor.ActorCell.systemInvoke(ActorCell.scala:535) at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:295) at akka.dispatch.Mailbox.run(Mailbox.scala:230) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:44:34,98 PM [system1-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:44:34,149 PM [system1-akka.actor.default-dispatcher-9] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-type-1: no initial generation override, starting from 0 01:44:34,149 PM [system1-akka.actor.default-dispatcher-9] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-type-1: persisting new identifier ClientIdentifier{frontend=member-1-frontend-type-1, generation=0} 01:44:34,149 PM [system1-akka.actor.default-dispatcher-9] [DEBUG] AbstractClientActor - member-1-frontend-type-1: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@6e34b067 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@484f596b 01:44:34,154 PM [system1-akka.actor.default-dispatcher-6] [DEBUG] SavingClientActorBehavior - member-1-frontend-type-1: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-type-1, 0, 1710164674150, None)) 01:44:34,154 PM [system1-akka.actor.default-dispatcher-6] [DEBUG] SavingClientActorBehavior - member-1-frontend-type-1: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164674149,0,0)) 01:44:34,155 PM [system1-akka.actor.default-dispatcher-6] [DEBUG] AbstractClientActor - member-1-frontend-type-1: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@484f596b to org.opendaylight.controller.cluster.access.client.ClientActorBehavior$MockitoMock$S7168poM@5f69462f 01:44:34,157 PM [system1-akka.actor.default-dispatcher-8] [ERROR] ActorBehaviorTest$MockedActor - Cannot invoke "org.opendaylight.controller.cluster.messaging.MessageAssembler.close()" because ".responseMessageAssembler" is null java.lang.NullPointerException: Cannot invoke "org.opendaylight.controller.cluster.messaging.MessageAssembler.close()" because ".responseMessageAssembler" is null at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.close(ClientActorBehavior.java:117) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.close(ClientActorBehavior.java:116) [ERROR] [03/11/2024 13:44:34.157] [system1-akka.actor.default-dispatcher-6] [akka://system1/user/$a] Cannot invoke "org.opendaylight.controller.cluster.messaging.MessageAssembler.close()" because ".responseMessageAssembler" is null java.lang.NullPointerException: Cannot invoke "org.opendaylight.controller.cluster.messaging.MessageAssembler.close()" because ".responseMessageAssembler" is null at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.close(ClientActorBehavior.java:117) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.close(ClientActorBehavior.java:116) at org.opendaylight.controller.cluster.access.client.AbstractClientActor.postStop(AbstractClientActor.java:37) at akka.actor.Actor.aroundPostStop(Actor.scala:556) at akka.actor.Actor.aroundPostStop$(Actor.scala:556) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundPostStop(PersistentActor.scala:295) at akka.persistence.Eventsourced.aroundPostStop(Eventsourced.scala:298) at akka.persistence.Eventsourced.aroundPostStop$(Eventsourced.scala:294) at akka.persistence.AbstractPersistentActor.aroundPostStop(PersistentActor.scala:295) at akka.actor.dungeon.FaultHandling.finishTerminate(FaultHandling.scala:242) at akka.actor.dungeon.FaultHandling.terminate(FaultHandling.scala:199) at akka.actor.dungeon.FaultHandling.terminate$(FaultHandling.scala:169) at akka.actor.ActorCell.terminate(ActorCell.scala:410) at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:519) at akka.actor.ActorCell.systemInvoke(ActorCell.scala:535) at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:295) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:273) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) at org.opendaylight.controller.cluster.access.client.AbstractClientActor.postStop(AbstractClientActor.java:37) at akka.actor.Actor.aroundPostStop(Actor.scala:556) at akka.actor.Actor.aroundPostStop$(Actor.scala:556) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundPostStop(PersistentActor.scala:295) at akka.persistence.Eventsourced.aroundPostStop(Eventsourced.scala:298) at akka.persistence.Eventsourced.aroundPostStop$(Eventsourced.scala:294) at akka.persistence.AbstractPersistentActor.aroundPostStop(PersistentActor.scala:295) at akka.actor.dungeon.FaultHandling.finishTerminate(FaultHandling.scala:242) at akka.actor.dungeon.FaultHandling.terminate(FaultHandling.scala:199) at akka.actor.dungeon.FaultHandling.terminate$(FaultHandling.scala:169) at akka.actor.ActorCell.terminate(ActorCell.scala:410) at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:519) at akka.actor.ActorCell.systemInvoke(ActorCell.scala:535) at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:295) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:273) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:44:34,161 PM [system1-akka.actor.default-dispatcher-9] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-type-1: recovered identifier ClientIdentifier{frontend=member-1-frontend-type-1, generation=0} 01:44:34,162 PM [system1-akka.actor.default-dispatcher-9] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-type-1: persisting new identifier ClientIdentifier{frontend=member-1-frontend-type-1, generation=1} 01:44:34,163 PM [system1-akka.actor.default-dispatcher-9] [DEBUG] AbstractClientActor - member-1-frontend-type-1: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@68338268 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4c70d4ba 01:44:34,163 PM [system1-akka.actor.default-dispatcher-7] [DEBUG] SavingClientActorBehavior - member-1-frontend-type-1: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-type-1, 0, 1710164674163, None)) 01:44:34,170 PM [system1-akka.actor.default-dispatcher-7] [DEBUG] SavingClientActorBehavior - member-1-frontend-type-1: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164674162,0,0)) 01:44:34,171 PM [system1-akka.actor.default-dispatcher-7] [DEBUG] AbstractClientActor - member-1-frontend-type-1: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4c70d4ba to org.opendaylight.controller.cluster.access.client.ClientActorBehavior$MockitoMock$S7168poM@5f69462f [INFO] [03/11/2024 13:44:34.176] [main] [CoordinatedShutdown(akka://system1)] Running CoordinatedShutdown with reason [ActorSystemTerminateReason] 01:44:34,177 PM [system1-akka.actor.default-dispatcher-8] [INFO] CoordinatedShutdown - Running CoordinatedShutdown with reason [ActorSystemTerminateReason] [ERROR] [03/11/2024 13:44:34.180] [system1-akka.actor.default-dispatcher-5] [akka://system1/user/$b] Cannot invoke "org.opendaylight.controller.cluster.messaging.MessageAssembler.close()" because ".responseMessageAssembler" is null java.lang.NullPointerException: Cannot invoke "org.opendaylight.controller.cluster.messaging.MessageAssembler.close()" because ".responseMessageAssembler" is null at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.close(ClientActorBehavior.java:117) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.close(ClientActorBehavior.java:116) at org.opendaylight.controller.cluster.access.client.AbstractClientActor.postStop(AbstractClientActor.java:37) at akka.actor.Actor.aroundPostStop(Actor.scala:556) at akka.actor.Actor.aroundPostStop$(Actor.scala:556) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundPostStop(PersistentActor.scala:295) at akka.persistence.Eventsourced.aroundPostStop(Eventsourced.scala:298) at akka.persistence.Eventsourced.aroundPostStop$(Eventsourced.scala:294) at akka.persistence.AbstractPersistentActor.aroundPostStop(PersistentActor.scala:295) at akka.actor.dungeon.FaultHandling.finishTerminate(FaultHandling.scala:242) at akka.actor.dungeon.FaultHandling.terminate(FaultHandling.scala:199) at akka.actor.dungeon.FaultHandling.terminate$(FaultHandling.scala:169) at akka.actor.ActorCell.terminate(ActorCell.scala:410) at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:519) at akka.actor.ActorCell.systemInvoke(ActorCell.scala:535) at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:295) at akka.dispatch.Mailbox.run(Mailbox.scala:230) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:44:34,181 PM [system1-akka.actor.default-dispatcher-8] [ERROR] ActorBehaviorTest$MockedActor - Cannot invoke "org.opendaylight.controller.cluster.messaging.MessageAssembler.close()" because ".responseMessageAssembler" is null java.lang.NullPointerException: Cannot invoke "org.opendaylight.controller.cluster.messaging.MessageAssembler.close()" because ".responseMessageAssembler" is null at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.close(ClientActorBehavior.java:117) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.close(ClientActorBehavior.java:116) at org.opendaylight.controller.cluster.access.client.AbstractClientActor.postStop(AbstractClientActor.java:37) at akka.actor.Actor.aroundPostStop(Actor.scala:556) at akka.actor.Actor.aroundPostStop$(Actor.scala:556) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundPostStop(PersistentActor.scala:295) at akka.persistence.Eventsourced.aroundPostStop(Eventsourced.scala:298) at akka.persistence.Eventsourced.aroundPostStop$(Eventsourced.scala:294) at akka.persistence.AbstractPersistentActor.aroundPostStop(PersistentActor.scala:295) at akka.actor.dungeon.FaultHandling.finishTerminate(FaultHandling.scala:242) at akka.actor.dungeon.FaultHandling.terminate(FaultHandling.scala:199) at akka.actor.dungeon.FaultHandling.terminate$(FaultHandling.scala:169) at akka.actor.ActorCell.terminate(ActorCell.scala:410) at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:519) at akka.actor.ActorCell.systemInvoke(ActorCell.scala:535) at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:295) at akka.dispatch.Mailbox.run(Mailbox.scala:230) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:44:34,239 PM [system1-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 01:44:34,282 PM [system1-akka.actor.default-dispatcher-7] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-type-1: no initial generation override, starting from 0 01:44:34,282 PM [system1-akka.actor.default-dispatcher-7] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-type-1: persisting new identifier ClientIdentifier{frontend=member-1-frontend-type-1, generation=0} 01:44:34,282 PM [system1-akka.actor.default-dispatcher-7] [DEBUG] AbstractClientActor - member-1-frontend-type-1: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@23ba1e54 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@135160f3 01:44:34,283 PM [system1-akka.actor.default-dispatcher-9] [DEBUG] SavingClientActorBehavior - member-1-frontend-type-1: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-type-1, 0, 1710164674282, None)) [WARN] [03/11/2024 13:44:34.285] [system1-akka.actor.default-dispatcher-9] [akka://system1/user/$a] received dead letter from Actor[akka://system1/system/in-memory-snapshot-store#1377119736]: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164674281,0,0)) 01:44:34,286 PM [system1-akka.actor.default-dispatcher-7] [INFO] RepointableActorRef - Message [akka.persistence.DeleteSnapshotsSuccess] from Actor[akka://system1/system/in-memory-snapshot-store#1377119736] to Actor[akka://system1/user/$a#1758170048] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://system1/user/$a#1758170048] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:44:34,289 PM [system1-akka.actor.default-dispatcher-11] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-type-1: no initial generation override, starting from 0 01:44:34,289 PM [system1-akka.actor.default-dispatcher-11] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-type-1: persisting new identifier ClientIdentifier{frontend=member-1-frontend-type-1, generation=0} 01:44:34,289 PM [system1-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientActor - member-1-frontend-type-1: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@30ae11e4 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2accb789 01:44:34,290 PM [system1-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-1-frontend-type-1: stashing command cmd 01:44:34,290 PM [system1-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-1-frontend-type-1: stashing command cmd 01:44:34,290 PM [system1-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-1-frontend-type-1: stashing command cmd 01:44:34,291 PM [system1-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-1-frontend-type-1: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-type-1, 0, 1710164674290, None)) 01:44:34,294 PM [system1-akka.actor.default-dispatcher-6] [DEBUG] SavingClientActorBehavior - member-1-frontend-type-1: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164674289,0,0)) [INFO] [akkaDeadLetter][03/11/2024 13:44:34.285] [system1-akka.actor.default-dispatcher-7] [akka://system1/user/$a] Message [akka.persistence.DeleteSnapshotsSuccess] from Actor[akka://system1/system/in-memory-snapshot-store#1377119736] to Actor[akka://system1/user/$a#1758170048] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://system1/user/$a#1758170048] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:44:34,305 PM [system1-akka.actor.default-dispatcher-6] [DEBUG] AbstractClientActor - member-1-frontend-type-1: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2accb789 to org.opendaylight.controller.cluster.access.client.ClientActorBehavior$MockitoMock$S7168poM@37922937 [INFO] [03/11/2024 13:44:34.332] [main] [CoordinatedShutdown(akka://system1)] Running CoordinatedShutdown with reason [ActorSystemTerminateReason] 01:44:34,334 PM [system1-akka.actor.default-dispatcher-13] [INFO] CoordinatedShutdown - Running CoordinatedShutdown with reason [ActorSystemTerminateReason] [ERROR] [03/11/2024 13:44:34.338] [system1-akka.actor.default-dispatcher-6] [akka://system1/user/$b] Cannot invoke "org.opendaylight.controller.cluster.messaging.MessageAssembler.close()" because ".responseMessageAssembler" is null java.lang.NullPointerException: Cannot invoke "org.opendaylight.controller.cluster.messaging.MessageAssembler.close()" because ".responseMessageAssembler" is null at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.close(ClientActorBehavior.java:117) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.close(ClientActorBehavior.java:116) at org.opendaylight.controller.cluster.access.client.AbstractClientActor.postStop(AbstractClientActor.java:37) at akka.actor.Actor.aroundPostStop(Actor.scala:556) at akka.actor.Actor.aroundPostStop$(Actor.scala:556) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundPostStop(PersistentActor.scala:295) at akka.persistence.Eventsourced.aroundPostStop(Eventsourced.scala:298) at akka.persistence.Eventsourced.aroundPostStop$(Eventsourced.scala:294) at akka.persistence.AbstractPersistentActor.aroundPostStop(PersistentActor.scala:295) at akka.actor.dungeon.FaultHandling.finishTerminate(FaultHandling.scala:242) at akka.actor.dungeon.FaultHandling.terminate(FaultHandling.scala:199) at akka.actor.dungeon.FaultHandling.terminate$(FaultHandling.scala:169) at akka.actor.ActorCell.terminate(ActorCell.scala:410) at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:519) at akka.actor.ActorCell.systemInvoke(ActorCell.scala:535) at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:295) at akka.dispatch.Mailbox.run(Mailbox.scala:230) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:44:34,339 PM [system1-akka.actor.default-dispatcher-6] [ERROR] ActorBehaviorTest$MockedActor - Cannot invoke "org.opendaylight.controller.cluster.messaging.MessageAssembler.close()" because ".responseMessageAssembler" is null java.lang.NullPointerException: Cannot invoke "org.opendaylight.controller.cluster.messaging.MessageAssembler.close()" because ".responseMessageAssembler" is null at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.close(ClientActorBehavior.java:117) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.close(ClientActorBehavior.java:116) at org.opendaylight.controller.cluster.access.client.AbstractClientActor.postStop(AbstractClientActor.java:37) at akka.actor.Actor.aroundPostStop(Actor.scala:556) at akka.actor.Actor.aroundPostStop$(Actor.scala:556) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundPostStop(PersistentActor.scala:295) at akka.persistence.Eventsourced.aroundPostStop(Eventsourced.scala:298) at akka.persistence.Eventsourced.aroundPostStop$(Eventsourced.scala:294) at akka.persistence.AbstractPersistentActor.aroundPostStop(PersistentActor.scala:295) at akka.actor.dungeon.FaultHandling.finishTerminate(FaultHandling.scala:242) at akka.actor.dungeon.FaultHandling.terminate(FaultHandling.scala:199) at akka.actor.dungeon.FaultHandling.terminate$(FaultHandling.scala:169) at akka.actor.ActorCell.terminate(ActorCell.scala:410) at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:519) at akka.actor.ActorCell.systemInvoke(ActorCell.scala:535) at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:295) at akka.dispatch.Mailbox.run(Mailbox.scala:230) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:44:34,386 PM [system1-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 01:44:34,437 PM [system1-akka.actor.default-dispatcher-6] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-type-1: no initial generation override, starting from 0 01:44:34,437 PM [system1-akka.actor.default-dispatcher-6] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-type-1: persisting new identifier ClientIdentifier{frontend=member-1-frontend-type-1, generation=0} 01:44:34,437 PM [system1-akka.actor.default-dispatcher-6] [DEBUG] AbstractClientActor - member-1-frontend-type-1: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@77cdb0a5 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2a4ff9a1 01:44:34,441 PM [system1-akka.actor.default-dispatcher-6] [DEBUG] SavingClientActorBehavior - member-1-frontend-type-1: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-type-1, 0, 1710164674437, None)) [WARN] [03/11/2024 13:44:34.444] [system1-akka.actor.default-dispatcher-11] [akka://system1/user/$a] received dead letter from Actor[akka://system1/system/in-memory-snapshot-store#-53871516]: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164674436,0,0)) [INFO] [akkaDeadLetter][03/11/2024 13:44:34.444] [system1-akka.actor.default-dispatcher-7] [akka://system1/user/$a] Message [akka.persistence.DeleteSnapshotsSuccess] from Actor[akka://system1/system/in-memory-snapshot-store#-53871516] to Actor[akka://system1/user/$a#1558258242] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://system1/user/$a#1558258242] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:44:34,449 PM [system1-akka.actor.default-dispatcher-6] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-type-1: no initial generation override, starting from 0 01:44:34,449 PM [system1-akka.actor.default-dispatcher-6] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-type-1: persisting new identifier ClientIdentifier{frontend=member-1-frontend-type-1, generation=0} 01:44:34,449 PM [system1-akka.actor.default-dispatcher-6] [DEBUG] AbstractClientActor - member-1-frontend-type-1: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@6b72f39a to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5ad31d17 01:44:34,452 PM [system1-akka.actor.default-dispatcher-8] [ERROR] SavingClientActorBehavior - member-1-frontend-type-1: failed to persist state java.lang.RuntimeException: save failed at org.opendaylight.controller.cluster.access.client.ActorBehaviorTest.testRecoveryAfterRestartSaveSnapshotFail(ActorBehaviorTest.java:141) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:728) at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60) at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131) at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156) at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147) at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:86) at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103) at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93) at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37) at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92) at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:218) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:214) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:139) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:69) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141) at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138) at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95) at java.base/java.util.ArrayList.forEach(ArrayList.java:1511) at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141) at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138) at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95) at java.base/java.util.ArrayList.forEach(ArrayList.java:1511) at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141) at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138) at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95) at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35) at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57) at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) 01:44:34,456 PM [system1-akka.actor.default-dispatcher-8] [DEBUG] AbstractClientActor - member-1-frontend-type-1: shutting down [INFO] [03/11/2024 13:44:34.460] [main] [CoordinatedShutdown(akka://system1)] Running CoordinatedShutdown with reason [ActorSystemTerminateReason] 01:44:34,461 PM [system1-akka.actor.default-dispatcher-8] [INFO] CoordinatedShutdown - Running CoordinatedShutdown with reason [ActorSystemTerminateReason]