question

max avatar image
max asked

When running multiple A/B Tests at the same time will the groups be fully random between each other?

If I run two A/B Tests 1) and 2), will about 50% of the users in 1)'s A be in 2)'s A and B ?

Or are Users always split the same way and when doing two A/B tests at the same time A and B are always the same Users?

Even if they are not the same users, is it safe that the users picked in two A/B tests are so random, that there is no skewing towards one or the other (e.g. 40% in 1)'s A are in 2)'s B

10 |1200

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

1 Answer

·
Sarah Zhang avatar image
Sarah Zhang answered

If you just create one A/B testing, and use it to do two tests. The users of A and B segments won't change. If you create two A/B testing settings, the users of the 2nd A/B testing settings won't be the same as the 1st one's. They will be redistributed and from a programming perspective, this process will be fully random. You can check the doc What is A/B testing? for more information about this feature in PlayFab.

10 |1200

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.