Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add actor testcontainer tests #1192

Open
wants to merge 18 commits into
base: master
Choose a base branch
from
Open

Conversation

akkie
Copy link

@akkie akkie commented Jan 16, 2025

Description

This PR adds testcontainer based integration tests for actors. This is to make sure that actors really work with testcontainers.

Issue reference

This PR was crated based on a Discord discussion with @salaboy to check if actors works with testcontainers.

@akkie akkie requested review from a team as code owners January 16, 2025 07:03
@akkie
Copy link
Author

akkie commented Jan 16, 2025

@salaboy I have currently the problem that the test fails with the following exception when trying to connect to the Dapr gRPC port.

Caused by: io.grpc.netty.shaded.io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: /127.0.0.1:50001
Caused by: java.net.ConnectException: Connection refused
        at java.base/sun.nio.ch.Net.pollConnect(Native Method)
        at java.base/sun.nio.ch.Net.pollConnectNow(Net.java:682)
        at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:1062)
        at io.grpc.netty.shaded.io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:336)
        at io.grpc.netty.shaded.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:339)
        at io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:776)
        at io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
        at io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
        at io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
        at io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:994)
        at io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Thread.java:1575)

Not sure why it uses port 50001, because this is the internal port. I think normally it should use the mapped port. Maybe you can have a look, if the test is not correct configured?

@salaboy
Copy link
Contributor

salaboy commented Jan 17, 2025

@akkie thanks a lot of this.. give me some time to look into this.. check the DCO, we need that to approve the PR. Click on the Details link to see the steps to fix it.

@akkie
Copy link
Author

akkie commented Jan 20, 2025

@salaboy Is it OK if I do a force push regarding the update of the DCO?

@salaboy
Copy link
Contributor

salaboy commented Jan 20, 2025

@akkie yeah.. that is your fork.. so it is ok

@akkie akkie force-pushed the actor-testcontainer branch from ee83cff to bec4bd2 Compare January 20, 2025 12:54
@salaboy
Copy link
Contributor

salaboy commented Jan 22, 2025

@akkie would you mind adding me as a collaborator to your fork? I am working on a fix, but I would love to push to your fork.. if not I can send you a patch to apply to your fork with the fix

@akkie
Copy link
Author

akkie commented Jan 22, 2025

@akkie would you mind adding me as a collaborator to your fork? I am working on a fix, but I would love to push to your fork.. if not I can send you a patch to apply to your fork with the fix

Done

@salaboy
Copy link
Contributor

salaboy commented Jan 22, 2025

@akkie I will push two commits to your fork, I am stuck with a new error now.. but at least the connection is working now.

@salaboy
Copy link
Contributor

salaboy commented Jan 22, 2025

Now I am stuck with this:

time="2025-01-22T17:41:56.55236659Z" level=debug msg="api error: code = Internal desc = error invoke actor method: did not find address for actor TestActor/f5893dd3-9d9c-4405-859e-b0c3b6aa2988" app_id=actor-dapr-app instance=780ae8017e66 scope=dapr.runtime.grpc.api type=log ver=1.14.1


io.dapr.exceptions.DaprException: INTERNAL: error invoke actor method: did not find address for actor TestActor/f5893dd3-9d9c-4405-859e-b0c3b6aa2988

But the connection is working as far as I can tell.

@salaboy
Copy link
Contributor

salaboy commented Jan 22, 2025

This is strange.. because it looks like we are hitting this: dapr/dapr#6783

https://github.com/dapr/java-sdk/blob/master/examples/src/main/java/io/dapr/examples/actors/DemoActorClient.java

@artursouza do you know if these examples are executed as part of the tests?

@akkie
Copy link
Author

akkie commented Jan 22, 2025

@salaboy This is exactly the message we get with our testcontainer setup:

fails to send binding event to http app channel, status code: 500 body: Dapr.DaprApiException: error invoke actor method: did not find address for actor

We use .NET and not Java.

@salaboy
Copy link
Contributor

salaboy commented Jan 22, 2025 via email

@salaboy
Copy link
Contributor

salaboy commented Jan 23, 2025

@akkie I am curious.. are you testing with an in-memory statestore?

@akkie
Copy link
Author

akkie commented Jan 23, 2025

@salaboy Yes, we are using Redis.

@salaboy
Copy link
Contributor

salaboy commented Jan 23, 2025

@akkie if you pull the code that I push can you check that you are getting the same results?

@akkie
Copy link
Author

akkie commented Jan 23, 2025

@salaboy You mean running the tests? If I run them, yes, I get the same result:

time="2025-01-23T15:27:19.585808881Z" level=debug msg="api error: code = Internal desc = error invoke actor method: did not find address for actor TestActor/7b7f7132-b3ab-4789-97d6-7dd9cfe6401d" app_id=actor-dapr-app instance=987309611b00 scope=dapr.runtime.grpc.api type=log ver=1.14.1

@salaboy
Copy link
Contributor

salaboy commented Jan 23, 2025

@akkie good news.. i think that I found the issue..
From what I can see there are two different things:

  1. We need to configure the app-port so the sidecar can contact back the application when it needs to execute an actor
  2. The actor runtime where we need to register actors (ActorRuntime.getInstance().registerActor(TestActorImpl.class);) is the one that is failing to get the right port. Because it is creating a new grpc channel without the overrides needed to connect to testcontainers..

I will try to fix this and push again.

@akkie
Copy link
Author

akkie commented Jan 30, 2025

@salaboy Any news regarding the issue?

@salaboy
Copy link
Contributor

salaboy commented Jan 30, 2025 via email

@salaboy
Copy link
Contributor

salaboy commented Jan 30, 2025 via email

@artur-ciocanu
Copy link
Contributor

@salaboy and @akkie I have found what's the culprit. Please check #1202. The TL;DR is that ActorRuntime doesn't allow properties override and that is why we see default GRPC port in logs and issues like actors not being found.

I will try to have a quick fix just to unblock this PR, but in general I think ActorRuntime should be aligned with WorkflowRuntime design for consistency sake.

CC: @artursouza @cicoyle

artur-ciocanu
artur-ciocanu previously approved these changes Feb 3, 2025
Copy link
Contributor

@artur-ciocanu artur-ciocanu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@akkie and @salaboy overall looks great, but there are some weird formatting issues that would be nice to address.

I think checkstyle will complain about it.

@akkie
Copy link
Author

akkie commented Feb 5, 2025

@artur-ciocanu Thanks for working on that. I fixed the formatting issues. Is there anything I need to do to get the tests running locally? They still failing for me with the message: api error: code = Internal desc = error invoke actor method: did not find address for actor TestActor/789215ad-9328-4019-8c86-6a4b6cc664a7

@salaboy
Copy link
Contributor

salaboy commented Feb 5, 2025

@akkie I think we are hitting the same issue as we hit with PubSub.. we need to investigate this further.. My expectation is that if we run this application in a Kubernetes Cluster, everything works.. so the issue is related with the sequence in which both the app and the sidecar gets bootstrapped by testcontainers.
I've created these examples for other features: #1208; we should add an actor Example there and run the applications locally to see if the issue exists outside of the test containers setup.

@salaboy
Copy link
Contributor

salaboy commented Feb 18, 2025 via email

@salaboy
Copy link
Contributor

salaboy commented Feb 20, 2025

@artursouza I couldn't reproduce the issue that you showed in the screenshot.. not sure how you run the test like that (multiple times without running the testcontainers setup). But now the test is fixed.

I've added some license headers to simplify the review process :)

@akkie apologies.. but you need to fix the DCO here..

@akkie
Copy link
Author

akkie commented Feb 20, 2025

@salaboy Tried to fix the DCO issue but there are some merge conflicts during the rebase. It's not so easy to fix them from my perspective because I have no idea how they should be resolved during the rebase. As example, the actual code looks completely different to what is suggested during conflict resolution. Do you have another idea how we can fix that?

❯ git rebase HEAD~15 --signoff
dropping be5530fdd71d16a65d984fe01ddb5fa59d383f19 Adding WorkflowTaskOptions and use it instead of TaskOptions (#1200) -- patch contents already upstream
dropping cf7405f97665ea9141b3118a6a279f0a51e13b42 adding spring boot workflows integration (#1195) -- patch contents already upstream
dropping 58d6218861e77a2588b0af360c978a25e5723091 Register workflows and acitivities using instances along classes (#1201) -- patch contents already upstream
dropping 0cec586d3552e5257d8f0399d267c42bc917c589 feat: Adding basic HTTPEndpoint configuration support in testcontainers module (#1210) -- patch contents already upstream
Auto-merging sdk-tests/src/test/java/io/dapr/it/spring/messaging/DaprSpringMessagingIT.java
CONFLICT (content): Merge conflict in sdk-tests/src/test/java/io/dapr/it/spring/messaging/DaprSpringMessagingIT.java
Auto-merging sdk-tests/src/test/java/io/dapr/it/spring/messaging/TestRestController.java
CONFLICT (content): Merge conflict in sdk-tests/src/test/java/io/dapr/it/spring/messaging/TestRestController.java
Auto-merging testcontainers-dapr/src/main/java/io/dapr/testcontainers/DaprContainer.java
CONFLICT (content): Merge conflict in testcontainers-dapr/src/main/java/io/dapr/testcontainers/DaprContainer.java
error: could not apply 22d9874a... Add app health check support to Dapr Testcontainer (#1213)
hint: Resolve all conflicts manually, mark them as resolved with
hint: "git add/rm <conflicted_files>", then run "git rebase --continue".
hint: You can instead skip this commit: run "git rebase --skip".
hint: To abort and get back to the state before "git rebase", run "git rebase --abort".
Could not apply 22d9874a... Add app health check support to Dapr Testcontainer (#1213)

Which is strange because there is no commit: 22d9874... Add app health check support to Dapr Testcontainer (#1213)

@salaboy
Copy link
Contributor

salaboy commented Feb 20, 2025

@akkie I am pinging you in discord

@akkie akkie force-pushed the actor-testcontainer branch 2 times, most recently from 6dab37f to 9485b77 Compare February 20, 2025 14:05
@artursouza
Copy link
Member

Let me try to fix this now.

@artur-ciocanu
Copy link
Contributor

artur-ciocanu commented Feb 20, 2025

@akkie I have looked at the failing tests and here is what we have:

Caused by: org.springframework.boot.web.server.PortInUseException: Port 8080 is already in use
	at org.springframework.boot.web.server.PortInUseException.lambda$throwIfPortBindingException$0(PortInUseException.java:70)
	at org.springframework.boot.web.server.PortInUseException.lambda$ifPortBindingException$1(PortInUseException.java:85)
	at org.springframework.boot.web.server.PortInUseException.ifCausedBy(PortInUseException.java:103)
	at org.springframework.boot.web.server.PortInUseException.ifPortBindingException(PortInUseException.java:82)
	at org.springframework.boot.web.server.PortInUseException.throwIfPortBindingException(PortInUseException.java:69)
	at org.springframework.boot.web.embedded.tomcat.TomcatWebServer.start(TomcatWebServer.java:250)
	at org.springframework.boot.web.servlet.context.WebServerStartStopLifecycle.start(WebServerStartStopLifecycle.java:44)
	at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:288)
	... 92 common frames omitted

To fix it we can use something like this:

/*
 * Copyright 2025 The Dapr Authors
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *     http://www.apache.org/licenses/LICENSE-2.0
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
limitations under the License.
*/

package io.dapr.it.testcontainers;

import io.dapr.actors.ActorId;
import io.dapr.actors.client.ActorClient;
import io.dapr.actors.client.ActorProxyBuilder;
import io.dapr.actors.runtime.ActorRuntime;
import io.dapr.testcontainers.Component;
import io.dapr.testcontainers.DaprContainer;
import io.dapr.testcontainers.DaprLogLevel;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Tag;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.test.context.SpringBootTest.WebEnvironment;
import org.springframework.test.context.DynamicPropertyRegistry;
import org.springframework.test.context.DynamicPropertySource;
import org.testcontainers.containers.Network;
import org.testcontainers.containers.wait.strategy.Wait;
import org.testcontainers.junit.jupiter.Container;
import org.testcontainers.junit.jupiter.Testcontainers;

import java.util.Map;
import java.util.Random;
import java.util.UUID;

import static org.junit.jupiter.api.Assertions.assertEquals;

@SpringBootTest(
    webEnvironment = WebEnvironment.RANDOM_PORT,
    classes = {
        TestActorsApplication.class,
        TestDaprActorsConfiguration.class
    }
)
@Testcontainers
@Tag("testcontainers")
public class DaprActorsIT {
  private static final Network DAPR_NETWORK = Network.newNetwork();
  private static final Random RANDOM = new Random();
  private static final int PORT = RANDOM.nextInt(1000) + 8000;

  private static final String ACTORS_MESSAGE_PATTERN = ".*Actor API level in the cluster has been updated to 10.*";

  @Container
  private static final DaprContainer DAPR_CONTAINER = new DaprContainer("daprio/daprd:1.14.4")
      .withAppName("actor-dapr-app")
      .withNetwork(DAPR_NETWORK)
      .withComponent(new Component("kvstore", "state.in-memory", "v1",
          Map.of("actorStateStore", "true")))
      .withDaprLogLevel(DaprLogLevel.DEBUG)
      .withLogConsumer(outputFrame -> System.out.println(outputFrame.getUtf8String()))
      .withAppChannelAddress("host.testcontainers.internal")
      .withAppPort(PORT);

  /**
   * Expose the Dapr ports to the host.
   *
   * @param registry the dynamic property registry
   */
  @DynamicPropertySource
  static void daprProperties(DynamicPropertyRegistry registry) {
    registry.add("dapr.http.endpoint", DAPR_CONTAINER::getHttpEndpoint);
    registry.add("dapr.grpc.endpoint", DAPR_CONTAINER::getGrpcEndpoint);
    registry.add("server.port", () -> PORT);
  }

  @Autowired
  private ActorClient daprActorClient;

  @Autowired
  private ActorRuntime daprActorRuntime;

  @BeforeEach
  public void setUp(){
    org.testcontainers.Testcontainers.exposeHostPorts(PORT);
    daprActorRuntime.registerActor(TestActorImpl.class);
    // Ensure the subscriptions are registered
    Wait.forLogMessage(ACTORS_MESSAGE_PATTERN, 1).waitUntilReady(DAPR_CONTAINER);
  }

  @Test
  public void testActors() {
    ActorProxyBuilder<TestActor> builder = new ActorProxyBuilder<>(TestActor.class, daprActorClient);
    ActorId actorId = ActorId.createRandom();
    TestActor actor = builder.build(actorId);

    String message = UUID.randomUUID().toString();

    String echoedMessage = actor.echo(message);

    assertEquals(echoedMessage, message);
  }
}

The key idea is that we use @DynamicPropertySource to override the port and we use Java Random to generate a random port and then use it for:

  • Dapr container app port
  • Dynamic property source override
  • Testcontainers expose port

Please give it a try and let me know.

CC: @salaboy @artursouza @cicoyle

@salaboy
Copy link
Contributor

salaboy commented Feb 21, 2025

@artursouza @akkie is fixed now.. but the damn DCO is blocking us to ren the pipelines again.. @akkie would you mind to signoff the latest commits?

@akkie akkie force-pushed the actor-testcontainer branch from 7b544c6 to 7483cd9 Compare February 21, 2025 08:56
@artur-ciocanu
Copy link
Contributor

@jakesmolka could you please the comments in this PR. As far as I remember you have mentioned that using Spring Boot Test with a defined port was problematic. I think we have found a way, it is not ideal, but it should work.

I wonder what are your thoughts.

akkie and others added 18 commits February 23, 2025 12:05
Signed-off-by: Christian Kaps <[email protected]>
Signed-off-by: Christian Kaps <[email protected]>
Signed-off-by: Christian Kaps <[email protected]>
Signed-off-by: Christian Kaps <[email protected]>
Signed-off-by: Christian Kaps <[email protected]>
Signed-off-by: Christian Kaps <[email protected]>
Co-authored-by: Cassie Coyle <[email protected]>
Signed-off-by: Christian Kaps <[email protected]>
…rs module (dapr#1210)

* feat: Adding basic HTTPEndpoint configuration support in testcontainers module

Signed-off-by: Laurent Broudoux <[email protected]>

* feat: dapr#1209 Adding test for HTTPEndpoint in testcontainers module

Signed-off-by: Laurent Broudoux <[email protected]>

---------

Signed-off-by: Laurent Broudoux <[email protected]>
Signed-off-by: Christian Kaps <[email protected]>
* Add app health check support to Dapr Testcontainer

Signed-off-by: Artur Ciocanu <[email protected]>

* Some minor cleanup

Signed-off-by: Artur Ciocanu <[email protected]>

* Move waiting to beforeEach, it looks more natural

Signed-off-by: Artur Ciocanu <[email protected]>

---------

Signed-off-by: Artur Ciocanu <[email protected]>
Co-authored-by: Artur Ciocanu <[email protected]>
Signed-off-by: Christian Kaps <[email protected]>
* Add app health check support to Dapr Testcontainer

Signed-off-by: Artur Ciocanu <[email protected]>

* Some minor cleanup

Signed-off-by: Artur Ciocanu <[email protected]>

* Move waiting to beforeEach, it looks more natural

Signed-off-by: Artur Ciocanu <[email protected]>

---------

Signed-off-by: Artur Ciocanu <[email protected]>
Co-authored-by: Artur Ciocanu <[email protected]>
Signed-off-by: Christian Kaps <[email protected]>
* Add app health check support to Dapr Testcontainer

Signed-off-by: Artur Ciocanu <[email protected]>

* Some minor cleanup

Signed-off-by: Artur Ciocanu <[email protected]>

* Move waiting to beforeEach, it looks more natural

Signed-off-by: Artur Ciocanu <[email protected]>

---------

Signed-off-by: Artur Ciocanu <[email protected]>
Co-authored-by: Artur Ciocanu <[email protected]>
Signed-off-by: Christian Kaps <[email protected]>
@akkie akkie force-pushed the actor-testcontainer branch from ace4137 to a9fdda8 Compare February 23, 2025 11:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants