question

admin-19 avatar image
admin-19 asked

Containerized Server Not Connectable

Our Unreal 4 game server is struggling with containerization. Running in process mode, it successfully sends heartbeats, goes active, and terminates. However, with Containerization mode on, LocalMultiplayerAgent.exe gets to "Waiting for heartbeats from the game server....." and does not progress.

I have run setup.ps1 with the designated port as 7777 instead of 56001, but before and after doing so was the same result.

We're unable to determine what is causing this issue. We were hopeful that it was simply an issue with the local containers or the LocalMultiplayerAgent but without being able to know for sure, we can't determine if the issues we are having while trying to connect to the server when it is utilized in a build are due to containerization or something in the connection process between ue4 and playfabs code.

When deployed, our server stays up, can be assigned as part of a queue, and switches to active mode when two players use it to connect, but the expected level does not load and we have no way of determining where that error is taking place. (Whether it's an issue with containerization of the server, or of the matchmaking process).

output-in-window.txtmultiplayersettings.txt

Custom Game Servers
10 |1200

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Seth Du avatar image
Seth Du answered

Please note that if a server build can run successfully in process mode while keeps waiting for heartbeat in container mode, the main causes are

  • lack of DLLs
  • not using server build of UE4
  • ports conflict.

In your scenario, I am guessing the port conflict. You should check that all existing containers have been removed (no matter in what state). If VM agent works successfully, it will kill the container process, export logs, then remove the current container, while if the process is shut down manually, it will keep the containers in Exit state. Due to sharing the same configuration JSON, all the containers will use the same port, which will cause issues.

Hence, please follow the steps below to troubleshoot:

  1. run Powershell in administrative mode
  2. Run -> docker ps -a. If there are any containers listed in the result, it will cause pending at waiting for heartbeats.
  3. According to Locally debugging game servers and integration with PlayFab - PlayFab | Microsoft Docs. run the command to remove all containers. You may also use -> docker rm [container ID] to do the same thing.
  4. In addition, you may also use docker
  5. Run the setup.ps1 if necessary, then run the vm agent again.

Hope this will solve your question. If you still have this issue, and you are in paid subscription plan, it is also recommended to submit a support ticket to share a sample project for us troubleshooting privately.

2 comments
10 |1200

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

admin-19 avatar image admin-19 commented ·

The instructions for properly connecting the dependencies are woefully incomplete, and the lack of official documentation for a very prominent use-case of your service is not abated by the existence of a different consumer's successful use of it.

We got it working, no thanks to the terrible lack of information available, however it's given us serious pause as to whether this is going to be the service we continue using. Every search for an answer leads to older and older instances of similar issues for different users, and it's terrifying to think of what kind of issues we may run into once the game is actually up and running, if documentation is this poor for even getting the system up at all.

1 Like 1 ·
Seth Du avatar image Seth Du ♦ commented ·

There are few other necessary steps when building the UE4 project.

  • disable WindowsMoviePlayer Plugin -- Pre-packaging Tip: denylisting or Removing UE4 Plugins
  • Because the UE4 project will create 2 exe processes, and each of them will have different DLL dependencies, hence you may need to add all DLLs. Please feel free to let me know if you have trouble in adding DLLs.
0 Likes 0 ·
Seth Du avatar image
Seth Du answered

“Waiting for heartbeats from the game server.....” means the server build is not initialized properly. Since you are using UE4, may I ask have you set up a server build target? It is same in the Unity Engine that a dedicated server needs a server build. Running in container mode will have the exact same working environment when it is uploaded to PlayFab. Please refer to Unreal official document -- Setting Up Dedicated Servers | Unreal Engine Documentation

8 comments
10 |1200

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

admin-19 avatar image admin-19 commented ·

Yes. I have set up a server build target, and I have also used the Huli documentation to walk through creation of a server package using Unreal Engine's Project Launcher tool to set up a customized build profile and ensure all the necessary elements were included in the build.

Again, please understand that the server build is working in process mode correctly. It obtains heartbeats, goes to active, then terminates without any issues along the way.

I have gone through and added all of the necessary DLL's to my folder, but we simply aren't getting heartbeats from the game server in container mode.

In process mode, yes. In container mode, no.

0 Likes 0 ·
Seth Du avatar image Seth Du ♦ admin-19 commented ·

Before we dig into this, have you referred the document on Connect directly to debug game servers - PlayFab | Microsoft Docs and call docker command to remove the leftover containers?

0 Likes 0 ·
admin-19 avatar image admin-19 Seth Du ♦ commented ·

There are no docker containers appearing when running "docker ps -a" and all containers are deleted in Docker Desktop.

0 Likes 0 ·
Show more comments
admin-19 avatar image admin-19 commented ·

@SethDu A lot of this is unnecessarily confusing and contradictory.

The example JSON at

https://docs.microsoft.com/en-us/gaming/playfab/features/multiplayer/servers/locally-debugging-game-servers-and-integration-with-playfab

has the Node Port and Agent Listening port mapped to the same value (56001). Editing the agent listening port to be 56001 (and running setup.ps1 again to make sure those firewalls are open) and then running the file again gets the same result. Running it with 56001 as the Listening Port and the Node port (as shown in the JSON) also yields the same result. Waiting for heartbeats.

Altering the file path to use the file in the binaries folder did indeed yield a different result. The console printed something that I couldn't read fast enough before closing, and it left a blank log in the GameLogs folder with nothing written in it. (As opposed to leaving no log when stuck at waiting for heartbeats, since logs are written and saved after the closing of the server).

0 Likes 0 ·
admin-19 avatar image admin-19 admin-19 commented ·

Tinkering some more and getting some logs now that are saying the recommended command path isn't recognized. Gonna keep playing with the JSON for a bit to see if I can't resolve that (since the file path for Binaries\win64\ProjectBlasterServer.exe is there and should be working).

0 Likes 0 ·
admin-19 avatar image admin-19 admin-19 commented ·

Path is recognized now, log prints

{"log":"Running Startup Script\n","stream":"stdout","time":"2021-05-22T00:29:08.3097077Z"}
{"log":"Certificates located at: C:\\GameCertificates\n","stream":"stdout","time":"2021-05-22T00:29:08.3342852Z"}

But the VM is very clearly closing out immediately after opening the server up.

0 Likes 0 ·

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.