01:14:59,837 AM [system1-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:15:00,596 AM [system1-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-type-1: no initial generation override, starting from 0 01:15:00,600 AM [system1-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-type-1: persisting new identifier ClientIdentifier{frontend=member-1-frontend-type-1, generation=0} 01:15:00,603 AM [system1-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-type-1: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@1fa5c3d4 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4527dbc7 01:15:00,613 AM [system1-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-1-frontend-type-1: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-type-1, 0, 1708737300605, None)) 01:15:00,617 AM [system1-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-1-frontend-type-1: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1708737300604,0,0)) 01:15:00,662 AM [system1-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-type-1: recovered identifier ClientIdentifier{frontend=another-frontend-type-2, generation=0} 01:15:00,664 AM [system1-akka.actor.default-dispatcher-10] [ERROR] RecoveringClientActorBehavior - member-1-frontend-type-1: Mismatched frontend identifier, shutting down. Current: member-1-frontend-type-1 Saved: another-frontend-type-2 01:15:00,664 AM [system1-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-type-1: shutting down 01:15:00,679 AM [system1-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-1-frontend-type-1: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4527dbc7 to org.opendaylight.controller.cluster.access.client.ClientActorBehavior$MockitoMock$IYXBwYly@7d66a126 01:15:00,697 AM [system1-akka.actor.default-dispatcher-5] [ERROR] ActorBehaviorTest$MockedActor - Cannot invoke "org.opendaylight.controller.cluster.messaging.MessageAssembler.close()" because ".responseMessageAssembler" is null java.lang.NullPointerException: Cannot invoke "org.opendaylight.controller.cluster.messaging.MessageAssembler.close()" because ".responseMessageAssembler" is null at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.close(ClientActorBehavior.java:117) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.close(ClientActorBehavior.java:116) at org.opendaylight.controller.cluster.access.client.AbstractClientActor.postStop(AbstractClientActor.java:37) at akka.actor.Actor.aroundPostStop(Actor.scala:556) at akka.actor.Actor.aroundPostStop$(Actor.scala:556) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundPostStop(PersistentActor.scala:295) at akka.persistence.Eventsourced.aroundPostStop(Eventsourced.scala:298) at akka.persistence.Eventsourced.aroundPostStop$(Eventsourced.scala:294) at akka.persistence.AbstractPersistentActor.aroundPostStop(PersistentActor.scala:295) at akka.actor.dungeon.FaultHandling.finishTerminate(FaultHandling.scala:242) at akka.actor.dungeon.FaultHandling.terminate(FaultHandling.scala:199) at akka.actor.dungeon.FaultHandling.terminate$(FaultHandling.scala:169) at akka.actor.ActorCell.terminate(ActorCell.scala:410) at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:519) at akka.actor.ActorCell.systemInvoke(ActorCell.scala:535) at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:295) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:273) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) [ERROR] [02/24/2024 01:15:00.688] [system1-akka.actor.default-dispatcher-4] [akka://system1/user/$a] Cannot invoke "org.opendaylight.controller.cluster.messaging.MessageAssembler.close()" because ".responseMessageAssembler" is null java.lang.NullPointerException: Cannot invoke "org.opendaylight.controller.cluster.messaging.MessageAssembler.close()" because ".responseMessageAssembler" is null at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.close(ClientActorBehavior.java:117) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.close(ClientActorBehavior.java:116) at org.opendaylight.controller.cluster.access.client.AbstractClientActor.postStop(AbstractClientActor.java:37) at akka.actor.Actor.aroundPostStop(Actor.scala:556) at akka.actor.Actor.aroundPostStop$(Actor.scala:556) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundPostStop(PersistentActor.scala:295) at akka.persistence.Eventsourced.aroundPostStop(Eventsourced.scala:298) at akka.persistence.Eventsourced.aroundPostStop$(Eventsourced.scala:294) at akka.persistence.AbstractPersistentActor.aroundPostStop(PersistentActor.scala:295) at akka.actor.dungeon.FaultHandling.finishTerminate(FaultHandling.scala:242) at akka.actor.dungeon.FaultHandling.terminate(FaultHandling.scala:199) at akka.actor.dungeon.FaultHandling.terminate$(FaultHandling.scala:169) at akka.actor.ActorCell.terminate(ActorCell.scala:410) at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:519) at akka.actor.ActorCell.systemInvoke(ActorCell.scala:535) at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:295) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:273) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) [INFO] [02/24/2024 01:15:00.703] [main] [CoordinatedShutdown(akka://system1)] Running CoordinatedShutdown with reason [ActorSystemTerminateReason] 01:15:00,705 AM [system1-akka.actor.default-dispatcher-4] [INFO] CoordinatedShutdown - Running CoordinatedShutdown with reason [ActorSystemTerminateReason] 01:15:00,794 AM [system1-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:15:00,851 AM [system1-akka.actor.default-dispatcher-7] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-type-1: no initial generation override, starting from 0 01:15:00,851 AM [system1-akka.actor.default-dispatcher-7] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-type-1: persisting new identifier ClientIdentifier{frontend=member-1-frontend-type-1, generation=0} 01:15:00,852 AM [system1-akka.actor.default-dispatcher-7] [DEBUG] AbstractClientActor - member-1-frontend-type-1: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@6b382129 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5a998d34 01:15:00,854 AM [system1-akka.actor.default-dispatcher-7] [DEBUG] SavingClientActorBehavior - member-1-frontend-type-1: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-type-1, 0, 1708737300853, None)) [WARN] [02/24/2024 01:15:00.858] [system1-akka.actor.default-dispatcher-6] [akka://system1/user/$a] received dead letter from Actor[akka://system1/system/in-memory-snapshot-store#-107663427]: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1708737300852,0,0)) 01:15:00,860 AM [system1-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-type-1: no initial generation override, starting from 0 01:15:00,861 AM [system1-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-type-1: persisting new identifier ClientIdentifier{frontend=member-1-frontend-type-1, generation=0} 01:15:00,861 AM [system1-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-type-1: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@b17de66 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3516a166 01:15:00,861 AM [system1-akka.actor.default-dispatcher-6] [DEBUG] SavingClientActorBehavior - member-1-frontend-type-1: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-type-1, 0, 1708737300861, None)) 01:15:00,863 AM [system1-akka.actor.default-dispatcher-10] [WARN] SavingClientActorBehavior - member-1-frontend-type-1: failed to delete prior snapshots java.lang.RuntimeException: delete failed at org.opendaylight.controller.cluster.access.client.ActorBehaviorTest.testRecoveryAfterRestartDeleteSnapshotsFail(ActorBehaviorTest.java:156) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:728) at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60) at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131) at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156) at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147) at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:86) at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103) at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93) at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37) at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92) at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:218) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:214) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:139) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:69) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141) at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138) at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95) at java.base/java.util.ArrayList.forEach(ArrayList.java:1511) at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141) at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138) at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95) at java.base/java.util.ArrayList.forEach(ArrayList.java:1511) at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141) at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138) at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95) at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35) at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57) at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) [INFO] [akkaDeadLetter][02/24/2024 01:15:00.865] [system1-akka.actor.default-dispatcher-7] [akka://system1/user/$a] Message [akka.persistence.DeleteSnapshotsSuccess] from Actor[akka://system1/system/in-memory-snapshot-store#-107663427] to Actor[akka://system1/user/$a#-1005845944] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://system1/user/$a#-1005845944] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) 01:15:00,868 AM [system1-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-type-1: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3516a166 to org.opendaylight.controller.cluster.access.client.ClientActorBehavior$MockitoMock$IYXBwYly@d7c00de 01:15:00,875 AM [system1-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [akka.persistence.DeleteSnapshotsSuccess] from Actor[akka://system1/system/in-memory-snapshot-store#-107663427] to Actor[akka://system1/user/$a#-1005845944] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://system1/user/$a#-1005845944] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:15:00,968 AM [system1-akka.actor.default-dispatcher-5] [INFO] CoordinatedShutdown - Running CoordinatedShutdown with reason [ActorSystemTerminateReason] [INFO] [02/24/2024 01:15:00.967] [main] [CoordinatedShutdown(akka://system1)] Running CoordinatedShutdown with reason [ActorSystemTerminateReason] [ERROR] [02/24/2024 01:15:00.972] [system1-akka.actor.default-dispatcher-10] [akka://system1/user/$b] Cannot invoke "org.opendaylight.controller.cluster.messaging.MessageAssembler.close()" because ".responseMessageAssembler" is null java.lang.NullPointerException: Cannot invoke "org.opendaylight.controller.cluster.messaging.MessageAssembler.close()" because ".responseMessageAssembler" is null at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.close(ClientActorBehavior.java:117) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.close(ClientActorBehavior.java:116) at org.opendaylight.controller.cluster.access.client.AbstractClientActor.postStop(AbstractClientActor.java:37) at akka.actor.Actor.aroundPostStop(Actor.scala:556) at akka.actor.Actor.aroundPostStop$(Actor.scala:556) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundPostStop(PersistentActor.scala:295) at akka.persistence.Eventsourced.aroundPostStop(Eventsourced.scala:298) at akka.persistence.Eventsourced.aroundPostStop$(Eventsourced.scala:294) at akka.persistence.AbstractPersistentActor.aroundPostStop(PersistentActor.scala:295) at akka.actor.dungeon.FaultHandling.finishTerminate(FaultHandling.scala:242) at akka.actor.dungeon.FaultHandling.terminate(FaultHandling.scala:199) at akka.actor.dungeon.FaultHandling.terminate$(FaultHandling.scala:169) at akka.actor.ActorCell.terminate(ActorCell.scala:410) at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:519) at akka.actor.ActorCell.systemInvoke(ActorCell.scala:535) at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:295) at akka.dispatch.Mailbox.run(Mailbox.scala:230) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:15:00,972 AM [system1-akka.actor.default-dispatcher-12] [ERROR] ActorBehaviorTest$MockedActor - Cannot invoke "org.opendaylight.controller.cluster.messaging.MessageAssembler.close()" because ".responseMessageAssembler" is null java.lang.NullPointerException: Cannot invoke "org.opendaylight.controller.cluster.messaging.MessageAssembler.close()" because ".responseMessageAssembler" is null at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.close(ClientActorBehavior.java:117) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.close(ClientActorBehavior.java:116) at org.opendaylight.controller.cluster.access.client.AbstractClientActor.postStop(AbstractClientActor.java:37) at akka.actor.Actor.aroundPostStop(Actor.scala:556) at akka.actor.Actor.aroundPostStop$(Actor.scala:556) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundPostStop(PersistentActor.scala:295) at akka.persistence.Eventsourced.aroundPostStop(Eventsourced.scala:298) at akka.persistence.Eventsourced.aroundPostStop$(Eventsourced.scala:294) at akka.persistence.AbstractPersistentActor.aroundPostStop(PersistentActor.scala:295) at akka.actor.dungeon.FaultHandling.finishTerminate(FaultHandling.scala:242) at akka.actor.dungeon.FaultHandling.terminate(FaultHandling.scala:199) at akka.actor.dungeon.FaultHandling.terminate$(FaultHandling.scala:169) at akka.actor.ActorCell.terminate(ActorCell.scala:410) at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:519) at akka.actor.ActorCell.systemInvoke(ActorCell.scala:535) at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:295) at akka.dispatch.Mailbox.run(Mailbox.scala:230) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:15:01,14 AM [system1-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:15:01,67 AM [system1-akka.actor.default-dispatcher-7] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-type-1: no initial generation override, starting from 0 01:15:01,67 AM [system1-akka.actor.default-dispatcher-7] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-type-1: persisting new identifier ClientIdentifier{frontend=member-1-frontend-type-1, generation=0} 01:15:01,67 AM [system1-akka.actor.default-dispatcher-7] [DEBUG] AbstractClientActor - member-1-frontend-type-1: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@7f63e3d2 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@455a60dd 01:15:01,68 AM [system1-akka.actor.default-dispatcher-6] [DEBUG] SavingClientActorBehavior - member-1-frontend-type-1: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-type-1, 0, 1708737301067, None)) 01:15:01,73 AM [system1-akka.actor.default-dispatcher-6] [DEBUG] SavingClientActorBehavior - member-1-frontend-type-1: stashing command cmd 01:15:01,74 AM [system1-akka.actor.default-dispatcher-6] [DEBUG] SavingClientActorBehavior - member-1-frontend-type-1: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1708737301066,0,0)) 01:15:01,74 AM [system1-akka.actor.default-dispatcher-6] [DEBUG] AbstractClientActor - member-1-frontend-type-1: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@455a60dd to org.opendaylight.controller.cluster.access.client.ClientActorBehavior$MockitoMock$IYXBwYly@441762b8 [INFO] [02/24/2024 01:15:01.088] [main] [CoordinatedShutdown(akka://system1)] Running CoordinatedShutdown with reason [ActorSystemTerminateReason] 01:15:01,89 AM [system1-akka.actor.default-dispatcher-6] [INFO] CoordinatedShutdown - Running CoordinatedShutdown with reason [ActorSystemTerminateReason] [ERROR] [02/24/2024 01:15:01.097] [system1-akka.actor.default-dispatcher-7] [akka://system1/user/$a] Cannot invoke "org.opendaylight.controller.cluster.messaging.MessageAssembler.close()" because ".responseMessageAssembler" is null java.lang.NullPointerException: Cannot invoke "org.opendaylight.controller.cluster.messaging.MessageAssembler.close()" because ".responseMessageAssembler" is null at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.close(ClientActorBehavior.java:117) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.close(ClientActorBehavior.java:116) at org.opendaylight.controller.cluster.access.client.AbstractClientActor.postStop(AbstractClientActor.java:37) at akka.actor.Actor.aroundPostStop(Actor.scala:556) at akka.actor.Actor.aroundPostStop$(Actor.scala:556) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundPostStop(PersistentActor.scala:295) at akka.persistence.Eventsourced.aroundPostStop(Eventsourced.scala:298) at akka.persistence.Eventsourced.aroundPostStop$(Eventsourced.scala:294) at akka.persistence.AbstractPersistentActor.aroundPostStop(PersistentActor.scala:295) at akka.actor.dungeon.FaultHandling.finishTerminate(FaultHandling.scala:242) at akka.actor.dungeon.FaultHandling.terminate(FaultHandling.scala:199) at akka.actor.dungeon.FaultHandling.terminate$(FaultHandling.scala:169) at akka.actor.ActorCell.terminate(ActorCell.scala:410) at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:519) at akka.actor.ActorCell.systemInvoke(ActorCell.scala:535) at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:295) at akka.dispatch.Mailbox.run(Mailbox.scala:230) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:15:01,98 AM [system1-akka.actor.default-dispatcher-7] [ERROR] ActorBehaviorTest$MockedActor - Cannot invoke "org.opendaylight.controller.cluster.messaging.MessageAssembler.close()" because ".responseMessageAssembler" is null java.lang.NullPointerException: Cannot invoke "org.opendaylight.controller.cluster.messaging.MessageAssembler.close()" because ".responseMessageAssembler" is null at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.close(ClientActorBehavior.java:117) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.close(ClientActorBehavior.java:116) at org.opendaylight.controller.cluster.access.client.AbstractClientActor.postStop(AbstractClientActor.java:37) at akka.actor.Actor.aroundPostStop(Actor.scala:556) at akka.actor.Actor.aroundPostStop$(Actor.scala:556) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundPostStop(PersistentActor.scala:295) at akka.persistence.Eventsourced.aroundPostStop(Eventsourced.scala:298) at akka.persistence.Eventsourced.aroundPostStop$(Eventsourced.scala:294) at akka.persistence.AbstractPersistentActor.aroundPostStop(PersistentActor.scala:295) at akka.actor.dungeon.FaultHandling.finishTerminate(FaultHandling.scala:242) at akka.actor.dungeon.FaultHandling.terminate(FaultHandling.scala:199) at akka.actor.dungeon.FaultHandling.terminate$(FaultHandling.scala:169) at akka.actor.ActorCell.terminate(ActorCell.scala:410) at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:519) at akka.actor.ActorCell.systemInvoke(ActorCell.scala:535) at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:295) at akka.dispatch.Mailbox.run(Mailbox.scala:230) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:15:01,136 AM [system1-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:15:01,192 AM [system1-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-type-1: no initial generation override, starting from 0 01:15:01,193 AM [system1-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-type-1: persisting new identifier ClientIdentifier{frontend=member-1-frontend-type-1, generation=0} 01:15:01,193 AM [system1-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-type-1: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@1368628c to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6f07eef3 01:15:01,196 AM [system1-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-type-1: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-type-1, 0, 1708737301193, None)) [WARN] [02/24/2024 01:15:01.198] [system1-akka.actor.default-dispatcher-10] [akka://system1/user/$a] received dead letter from Actor[akka://system1/system/in-memory-snapshot-store#1923433125]: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1708737301192,0,0)) [INFO] [akkaDeadLetter][02/24/2024 01:15:01.199] [system1-akka.actor.default-dispatcher-10] [akka://system1/user/$a] Message [akka.persistence.DeleteSnapshotsSuccess] from Actor[akka://system1/system/in-memory-snapshot-store#1923433125] to Actor[akka://system1/user/$a#-244292782] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://system1/user/$a#-244292782] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:15:01,200 AM [system1-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [akka.persistence.DeleteSnapshotsSuccess] from Actor[akka://system1/system/in-memory-snapshot-store#1923433125] to Actor[akka://system1/user/$a#-244292782] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://system1/user/$a#-244292782] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:15:01,203 AM [system1-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-type-1: recovered identifier ClientIdentifier{frontend=member-1-frontend-type-1, generation=0} 01:15:01,205 AM [system1-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-type-1: persisting new identifier ClientIdentifier{frontend=member-1-frontend-type-1, generation=1} 01:15:01,205 AM [system1-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-type-1: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@2e3ae4c8 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6d155fa4 01:15:01,206 AM [system1-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-type-1: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-type-1, 0, 1708737301205, None)) [INFO] [02/24/2024 01:15:01.228] [main] [CoordinatedShutdown(akka://system1)] Running CoordinatedShutdown with reason [ActorSystemTerminateReason] 01:15:01,230 AM [system1-akka.actor.default-dispatcher-5] [INFO] CoordinatedShutdown - Running CoordinatedShutdown with reason [ActorSystemTerminateReason] [WARN] [02/24/2024 01:15:01.232] [system1-akka.actor.default-dispatcher-6] [akka://system1/user/$b] received dead letter from Actor[akka://system1/system/in-memory-snapshot-store#1923433125]: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1708737301204,0,0)) 01:15:01,286 AM [system1-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:15:01,337 AM [system1-akka.actor.default-dispatcher-7] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-type-1: no initial generation override, starting from 0 01:15:01,338 AM [system1-akka.actor.default-dispatcher-7] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-type-1: persisting new identifier ClientIdentifier{frontend=member-1-frontend-type-1, generation=0} 01:15:01,338 AM [system1-akka.actor.default-dispatcher-7] [DEBUG] AbstractClientActor - member-1-frontend-type-1: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@3b0328c5 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4107f0e1 01:15:01,340 AM [system1-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-type-1: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-type-1, 0, 1708737301339, None)) [WARN] [02/24/2024 01:15:01.343] [system1-akka.actor.default-dispatcher-7] [akka://system1/user/$a] received dead letter from Actor[akka://system1/system/in-memory-snapshot-store#1744780913]: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1708737301338,0,0)) 01:15:01,347 AM [system1-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-type-1: no initial generation override, starting from 0 01:15:01,347 AM [system1-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-type-1: persisting new identifier ClientIdentifier{frontend=member-1-frontend-type-1, generation=0} 01:15:01,348 AM [system1-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-type-1: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@193f3b3f to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7551d09c 01:15:01,348 AM [system1-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-type-1: stashing command cmd 01:15:01,348 AM [system1-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-type-1: stashing command cmd 01:15:01,348 AM [system1-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-type-1: stashing command cmd 01:15:01,349 AM [system1-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-type-1: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-type-1, 0, 1708737301349, None)) [INFO] [akkaDeadLetter][02/24/2024 01:15:01.354] [system1-akka.actor.default-dispatcher-6] [akka://system1/user/$a] Message [akka.persistence.DeleteSnapshotsSuccess] from Actor[akka://system1/system/in-memory-snapshot-store#1744780913] to Actor[akka://system1/user/$a#589049008] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://system1/user/$a#589049008] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:15:01,355 AM [system1-akka.actor.default-dispatcher-6] [INFO] RepointableActorRef - Message [akka.persistence.DeleteSnapshotsSuccess] from Actor[akka://system1/system/in-memory-snapshot-store#1744780913] to Actor[akka://system1/user/$a#589049008] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://system1/user/$a#589049008] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:15:01,356 AM [system1-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-type-1: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1708737301348,0,0)) 01:15:01,357 AM [system1-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-type-1: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7551d09c to org.opendaylight.controller.cluster.access.client.ClientActorBehavior$MockitoMock$IYXBwYly@7e1ce99c [INFO] [02/24/2024 01:15:01.380] [main] [CoordinatedShutdown(akka://system1)] Running CoordinatedShutdown with reason [ActorSystemTerminateReason] 01:15:01,381 AM [system1-akka.actor.default-dispatcher-12] [INFO] CoordinatedShutdown - Running CoordinatedShutdown with reason [ActorSystemTerminateReason] [ERROR] [02/24/2024 01:15:01.384] [system1-akka.actor.default-dispatcher-11] [akka://system1/user/$b] Cannot invoke "org.opendaylight.controller.cluster.messaging.MessageAssembler.close()" because ".responseMessageAssembler" is null java.lang.NullPointerException: Cannot invoke "org.opendaylight.controller.cluster.messaging.MessageAssembler.close()" because ".responseMessageAssembler" is null at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.close(ClientActorBehavior.java:117) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.close(ClientActorBehavior.java:116) at org.opendaylight.controller.cluster.access.client.AbstractClientActor.postStop(AbstractClientActor.java:37) at akka.actor.Actor.aroundPostStop(Actor.scala:556) at akka.actor.Actor.aroundPostStop$(Actor.scala:556) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundPostStop(PersistentActor.scala:295) at akka.persistence.Eventsourced.aroundPostStop(Eventsourced.scala:298) at akka.persistence.Eventsourced.aroundPostStop$(Eventsourced.scala:294) at akka.persistence.AbstractPersistentActor.aroundPostStop(PersistentActor.scala:295) at akka.actor.dungeon.FaultHandling.finishTerminate(FaultHandling.scala:242) at akka.actor.dungeon.FaultHandling.terminate(FaultHandling.scala:199) at akka.actor.dungeon.FaultHandling.terminate$(FaultHandling.scala:169) at akka.actor.ActorCell.terminate(ActorCell.scala:410) at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:519) at akka.actor.ActorCell.systemInvoke(ActorCell.scala:535) at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:295) at akka.dispatch.Mailbox.run(Mailbox.scala:230) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:15:01,385 AM [system1-akka.actor.default-dispatcher-11] [ERROR] ActorBehaviorTest$MockedActor - Cannot invoke "org.opendaylight.controller.cluster.messaging.MessageAssembler.close()" because ".responseMessageAssembler" is null java.lang.NullPointerException: Cannot invoke "org.opendaylight.controller.cluster.messaging.MessageAssembler.close()" because ".responseMessageAssembler" is null at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.close(ClientActorBehavior.java:117) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.close(ClientActorBehavior.java:116) at org.opendaylight.controller.cluster.access.client.AbstractClientActor.postStop(AbstractClientActor.java:37) at akka.actor.Actor.aroundPostStop(Actor.scala:556) at akka.actor.Actor.aroundPostStop$(Actor.scala:556) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundPostStop(PersistentActor.scala:295) at akka.persistence.Eventsourced.aroundPostStop(Eventsourced.scala:298) at akka.persistence.Eventsourced.aroundPostStop$(Eventsourced.scala:294) at akka.persistence.AbstractPersistentActor.aroundPostStop(PersistentActor.scala:295) at akka.actor.dungeon.FaultHandling.finishTerminate(FaultHandling.scala:242) at akka.actor.dungeon.FaultHandling.terminate(FaultHandling.scala:199) at akka.actor.dungeon.FaultHandling.terminate$(FaultHandling.scala:169) at akka.actor.ActorCell.terminate(ActorCell.scala:410) at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:519) at akka.actor.ActorCell.systemInvoke(ActorCell.scala:535) at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:295) at akka.dispatch.Mailbox.run(Mailbox.scala:230) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:15:01,432 AM [system1-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:15:01,477 AM [system1-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-type-1: no initial generation override, starting from 0 01:15:01,477 AM [system1-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-type-1: persisting new identifier ClientIdentifier{frontend=member-1-frontend-type-1, generation=0} 01:15:01,477 AM [system1-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-type-1: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@7c2e04b4 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7b7cde00 01:15:01,478 AM [system1-akka.actor.default-dispatcher-7] [DEBUG] SavingClientActorBehavior - member-1-frontend-type-1: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-type-1, 0, 1708737301478, None)) [WARN] [02/24/2024 01:15:01.481] [system1-akka.actor.default-dispatcher-5] [akka://system1/user/$a] received dead letter from Actor[akka://system1/system/in-memory-snapshot-store#-9968157]: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1708737301477,0,0)) 01:15:01,490 AM [system1-akka.actor.default-dispatcher-11] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-type-1: no initial generation override, starting from 0 01:15:01,490 AM [system1-akka.actor.default-dispatcher-11] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-type-1: persisting new identifier ClientIdentifier{frontend=member-1-frontend-type-1, generation=0} 01:15:01,490 AM [system1-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientActor - member-1-frontend-type-1: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@76116c98 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7372c77c 01:15:01,492 AM [system1-akka.actor.default-dispatcher-6] [INFO] RepointableActorRef - Message [akka.persistence.DeleteSnapshotsSuccess] from Actor[akka://system1/system/in-memory-snapshot-store#-9968157] to Actor[akka://system1/user/$a#757132578] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://system1/user/$a#757132578] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. [INFO] [akkaDeadLetter][02/24/2024 01:15:01.492] [system1-akka.actor.default-dispatcher-6] [akka://system1/user/$a] Message [akka.persistence.DeleteSnapshotsSuccess] from Actor[akka://system1/system/in-memory-snapshot-store#-9968157] to Actor[akka://system1/user/$a#757132578] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://system1/user/$a#757132578] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:15:01,494 AM [system1-akka.actor.default-dispatcher-12] [ERROR] SavingClientActorBehavior - member-1-frontend-type-1: failed to persist state java.lang.RuntimeException: save failed at org.opendaylight.controller.cluster.access.client.ActorBehaviorTest.testRecoveryAfterRestartSaveSnapshotFail(ActorBehaviorTest.java:141) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:728) at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60) at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131) at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156) at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147) at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:86) at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103) at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93) at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37) at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92) at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:218) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:214) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:139) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:69) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141) at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138) at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95) at java.base/java.util.ArrayList.forEach(ArrayList.java:1511) at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141) at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138) at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95) at java.base/java.util.ArrayList.forEach(ArrayList.java:1511) at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141) at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138) at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95) at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35) at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57) at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) 01:15:01,497 AM [system1-akka.actor.default-dispatcher-12] [DEBUG] AbstractClientActor - member-1-frontend-type-1: shutting down [INFO] [02/24/2024 01:15:01.500] [main] [CoordinatedShutdown(akka://system1)] Running CoordinatedShutdown with reason [ActorSystemTerminateReason] 01:15:01,502 AM [system1-akka.actor.default-dispatcher-12] [INFO] CoordinatedShutdown - Running CoordinatedShutdown with reason [ActorSystemTerminateReason]