question

robert avatar image
robert asked

Error: ​Maximum number of concurrent bulk segment jobs are already running

We recently always receive "Maximum number of concurrent bulk segment jobs are already running" when we try to run a task manually or periodically.

We have 3 tasks configured. No other task is running when we get the above error for a just manually launched task.

TitleID: 4D83

Result of such a task:

The segment this task should be executed for:

Hope you have any ideas, Thanks!

10 |1200

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

1 Answer

·
brendan avatar image
brendan answered

Three questions:

What specific task are you attempting to run when you get this error?

When (specifically) was the last time you got this error?

Do you mind if we test running the same task in your title, so that we can monitor the results?

16 comments
10 |1200

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

robert avatar image robert commented ·

Task in question is "SendPushOnFreeCoinsOrDailySpinAvailable", 116EC3449D068B7D

Just a few seconds ago.

This time we received "Unknown error occurred while querying players";

Interestingly this task instance "116EC3449D068B7D-49" is now in "Failed" State but the abort button is still active in the task instance details. Trying to run the task again immediately leads to the "Maximum number of concurrent bulk segment jobs are already running" error, maybe because the task still "hangs" in an active state although it failed already?

Please feel free to launch the task.

Thanks!

0 Likes 0 ·
robert avatar image robert commented ·

any ideas to the unknown error occured while querying players error on this specific task?

0 Likes 0 ·
brendan avatar image brendan robert commented ·

We're checking on this and we'll update as soon as we have more info.

0 Likes 0 ·
robert avatar image robert brendan commented ·

If there is any other way to get this tasks running quickly I would really appreciate it. its an rather important feature for us to keep the players playing, especially while being featured in the stores.

0 Likes 0 ·
Show more comments
robert avatar image robert commented ·

This is not really what we are seeing.

No task is running. (next task is schedule to be run in 6 hours for example).

We manually start this task.

After 5 seconds the task overview page shows "Failed"

Clicking on the task shows error: "Unknown error occurred while querying players". (The Abort button is still clickable)

Looks like the task is still running but the overview page shows failed already.

Of course, running another task now failed because the first one is not really finished although overview page shows Failed.

The problem is, why does the first task abort? (it never finishes successfully).

0 Likes 0 ·
brendan avatar image brendan robert commented ·

Interesting - what I'm seeing is that, according to their details, the tasks that hit the unknown error are failing, but only after a long time. That was actually item 2 in my info above. That's the part of this we're focusing on at the moment, since that does seem to be the key issue. We're not seeing this on any other title, and it doesn't repro in quick tests in my own title, which is why this is taking longer than usual.

0 Likes 0 ·
brendan avatar image brendan robert commented ·

Okay, very sorry for the delay on this, but the fix is now live. Please go ahead and re-try, and let us know what you see.

0 Likes 0 ·
robert avatar image robert brendan commented ·

Just gave it a try. This time it started and even processed players but eventually aborted again with following error:

{
    "TaskInstanceId": "116EC3449D068B7D-52",
    "TaskIdentifier": {
        "Name": null,
        "Id": "116EC3449D068B7D"
    },
    "StartedAt": "2017-05-30T06:28:04.0489668",
    "CompletedAt": "2017-05-30T15:28:04.0891813",
    "Status": "Aborted",
    "PercentComplete": 35.47,
    "EstimatedSecondsRemaining": 152477.98,
    "ScheduledByUserId": "8AAEAF658F91B1AE",
    "ErrorMessage": "Segment batch job failed with unknown cause",
    "ErrorWasFatal": false,
    "TotalPlayersInSegment": 73281,
    "TotalPlayersProcessed": 26000
}
0 Likes 0 ·
Show more comments
robert avatar image robert commented ·

Unfortunately we still can not really use the task system. There is a task "hanging" in the "aborting" state for more than 10 days already, no way to start a new task (which we would really need before the weekend starts to send out push notifications for an upcoming event)...

We filed a support ticket for this issue already, additional details in the ticket and slack channel.

Thanks

0 Likes 0 ·
brendan avatar image brendan robert commented ·

We've taken care of this issue, and responded in both Slack and the ticket. You shouldn't run into this anymore - the issue is a rare corner case, and we'll be updating with a fix shortly.

0 Likes 0 ·
robert avatar image robert brendan commented ·

Thanks for the quick response. For some reason it looks like that the task still doesn't run correctly (the last scheduled task after the fix aborted with: "Segment batch job failed with unknown cause").

I just scheduled another task and will let you know if this one fails too. Thanks!

0 Likes 0 ·

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.