question

Ruben Pascual Blanco avatar image
Ruben Pascual Blanco asked

Unhealthy Linux container build. Connected to remote VMachine, messed around with the container, seems to be a permissions problem ("operation not permitted")

This build works fine locally. After a lot of debugging attempts (it always shows "pending heartbeats" in the uploaded build but runs fine in LMA), I ended realizing that the process didn't even start inside the machine, after I managed to connect to the remote virtual machine, go inside the container trough docker exec, and test on it.

I tried running another instance of the process under strace both in localagent and the uploaded build. localagent runs fine but the uploaded build stops here:


As you can see everything goes wrong after a "operation not permitted" and this was the reason why the uploaded build never ever got to create a log file like it did in the LocalAgent container despite the process being running: the process got stuck here and went to sleep. At first I thought it may be the process crashing or taking too much CPU, but neither the server gets killed (a complete crash kills the server completely and you can't go inside it remotely), and neither CPU goes upwards of 0% nor the ram upwards of 200MB. CPU only gives a tiny "bip" when trying to launch as seen in VMetrics.

What can be causing this problem with permissions (or whatever its causing this operation not permitted) in the uploaded container but not while using the same image in the LocalMultiplayerAgent test? I connected both to the remote container and the local container trough the same SSH command to perform the testing.

multiplayer
processstrace.png (91.2 KiB)
1 comment
10 |1200

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

JayZuo avatar image JayZuo ♦ commented ·

May I know what's the Linux version you are using in your Dockerfile? Are you using Ubuntu 22.04 like the following?

FROM ubuntu:22.04
0 Likes 0 ·
Austin Brunkhorst avatar image
Austin Brunkhorst answered

Hi @Ruben Pascual Blanco,

Thanks for taking the time to explain your issue and provide the strace output. As @Jay Zuo mentioned, I suspect this may be related to using Ubuntu 22.04 as a base image, which is not currently supported for Linux containers on PlayFab. PlayFab Linux containers are using an older version of the docker runtime which has a different default seccomp policy (as @Gosen Gao noted). We do plan to offer an updated version of the docker runtime shipped with PlayFab Linux containers, but we do not yet have a date for when you can expect that.

However, we do have a short term workaround that you can try.

Add an asset with a startup script that allows all syscalls on docker
This does come with the obligatory warning that this is a workaround and exposes the virtual machine running your game servers to the risks associated with disabling seccomp. That said, it shouldn't be much of an issue.


1. Create a shell script named PF_StartupScript.sh with the following commands and add it to a zip archive OR use this one we've already tested: fixubuntu2004startupscript.zip

unconfinedProfile=/etc/docker/unconfinedSeccompProfile.json
echo '{"defaultAction": "SCMP_ACT_ALLOW"}' | sudo tee $unconfinedProfile
eval echo '{\"userland-proxy\": false, \"seccomp-profile\": \"$unconfinedProfile\"}' | sudo tee /etc/docker/daemon.json
sudo systemctl restart docker

2. Add the zip archive as an asset to your build. Use /mnt for the mount path

Using developer.playfab.com



Or using the REST API

"GameAssetReferences": [
    { "FileName": "FixUbuntu20.04StartupScript.zip", "MountPath": "/mnt" }
],

10 |1200

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Gosen Gao avatar image
Gosen Gao answered

I'm not an expert on Docker. But it seems like the clone3 function is not in the allow list of the default Docker profile, and that's maybe the reason why you got the "operation not permitted" error.

Referring to Docker docs -- Pass a profile for a container, which mentions “In effect, the profile is a allowlist which denies access to system calls by default, then allowlists specific system calls. The profile defines a specific list of system calls which are fully allowed, because their action is overridden to be SCMP_ACT_ALLOW.”

2 comments
10 |1200

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Ruben Pascual Blanco avatar image Ruben Pascual Blanco commented ·

Can this be happening due to Unity trying to create a PlayerPrefs file outside the assets folder? (in .config)? I saw that Clone3 is about creating child processes. Can be Unity doing this by default somehow? I don't create any child process deliberately on my server. Is it the console of the dedicated server build?

Can this be avoided? Or can the profile be disabled?

0 Likes 0 ·
Gosen Gao avatar image Gosen Gao Ruben Pascual Blanco commented ·

I don't know the details of Unity's internal operation, you may consult Unity for professional support. You can also check your project to see if any third-party assets make this call.

0 Likes 0 ·
hassanhnasser avatar image
hassanhnasser answered

Could you find a solution to this issue ?

1 comment
10 |1200

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Ruben Pascual Blanco avatar image Ruben Pascual Blanco commented ·

Not yet. It seems to be related to Unity creating a subprocess, which the virtual machine permissions deny so the app goes into sleep mode before loading.

Still researching which kind of subprocess would this be.

0 Likes 0 ·

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Error rendering WebPanel (widgets/consolidation-widget.ftl): org.hibernate.hql.internal.ast.QuerySyntaxException: AvailableConsolidation is not mapped [from AvailableConsolidation up where up.node = :node]