内部查看：Unity的场景测试

如 Unity Engine QA Process网志文章所述 ，每个Unity版本在发布之前都要经过多个测试阶段。 在这里，我想介绍一下其中的一个： 场景测试周 (在之前的博文中称为FTP v2)。 (As outlined in the Unity Engine QA Process blogpost, every Unity release goes through a number of test phases before it is released. Here I would like to dive in and present one of them: Scenario Test Week (dubbed FTP v2 in the previous blogpost).)

Simply put, it’s testing by creating games. We prepare for it by creating a number of scenarios – a short pitch for a game project. Examples could be “3D Racing Game”, “2D Platformer” and so on. Then we group participants into teams, assign each team a scenario and then the teams have one week to create a game based on that simple premise.

简而言之，它通过创建游戏进行测试。 我们通过创建许多方案为游戏项目做准备，以作准备。 例如“ 3D Racing Game”，“ 2D Platformer”等。 然后，我们将参与者分为小组，为每个小组分配一个方案，然后小组有一个星期的时间根据这一简单前提创建游戏。

For testing our most recent release (at this time of writing), Unity 5.5, we had 14 scenarios and 53 participants, including testers, student workers and developers.

为了测试我们最新的发行版(在撰写本文时)Unity 5.5，我们有14个场景和53个参与者，其中包括测试人员，学生工作者和开发人员。

But scenario testing is more than a game jam. It’s a testing phase with a very specific goal: Find bugs by using the Editor in a credible way. We have made a breakdown of Editor features and platforms and mapped it to the different scenarios to get as wide coverage as possible. We make sure that features are covered by more than one scenario. By having scenarios be different game genres, we ensure that the features are used in different ways.  

但是，场景测试不仅仅只是一场游戏。 这是一个具有特定目标的测试阶段：以可靠的方式使用编辑器查找错误。 我们对编辑器功能和平台进行了细分，并将其映射到不同的场景，以实现尽可能广泛的覆盖范围。 我们确保功能涵盖多个场景。 通过使场景成为不同的游戏类型，我们确保以不同的方式使用功能。

Areas mapped to different scenarios/teams for 5.5 Scenario Test Week

5.5场景测试周映射到不同场景/团队的区域

Different test approaches find different problems. We have more than 60000 automated tests with millions of executions each month. Our manual regression suite – Full Test Pass – consists of thousands of manual test-cases that provide a comprehensive sweep of the functionality in Unity. This is all great and invaluable! But broadly speaking, these kinds of tests tend to exercise functionality in isolation. Some errors appear when you combine features. Or they appear over time. Something might work the first five times you do it, only to fail the sixth. Furthermore – again, broadly speaking – test automation and manual test-cases repeat the same test over and over again. And while the scope of what we achieve with automation is impressive, it’s still only a tiny fraction of the theoretical possibility space for how Unity can be interacted with.

不同的测试方法会发现不同的问题。 我们 每月 有 超过60000项自动测试 ，执行了数百万次执行。 我们的手动回归套件-完整测试通过-包含数千个手动测试用例，它们提供了Unity中功能的全面概览。 这一切都是伟大而无价的！ 但是从广义上讲，这些测试往往会孤立地行使功能。 合并功能时会出现一些错误。 否则它们会随着时间出现。 前五次可能会起作用，但第六次失败。 此外，从广义上讲，测试自动化和手动测试用例一次又一次地重复相同的测试。 尽管我们通过自动化实现的范围令人印象深刻，但它仍然只是与Unity进行交互的理论可能性空间的一小部分。

With Scenario Testing we address some of these shortcomings. When you create a game, you don’t simply methodically go over each feature of a product. You get repetition of actions, you get randomness and the Editor will be interacted with in ways that are impossible to predict when designing a feature or even a set of tests. And most importantly, the Editor is used in a way that is credible. Even if it’s an approximation, we get to feel some of the joy and frustration that our end-users feel. This is powerful. Chomping on your own dog-food is always a good thing…

通过方案测试，我们可以解决其中的一些缺点。 在创建游戏时，您不会简单地有条不紊地浏览产品的每个功能。 您将获得重复的动作，将获得随机性，并且在设计功能甚至测试集时，编辑器将以无法预测的方式进行交互。 最重要的是，编辑器的使用方式是可靠的。 即使只是近似值，我们也要感受到最终用户所感到的喜悦和沮丧。 这很强大。 自己吃狗粮总是一件好事……

Bugs reported during 5.5 Scenario Test Week

5.5场景测试周期间报告的错误

Here is an example of the kinds of feedback yielded by scenario testing. One of the scenarios was to create a Real Time Strategy game (screenshot below). It was intended to obtain coverage for AI, Particles and Terrain. However, once the game was taking shape, naturally the idea came up to add multiplayer. And very quickly the team ran into problems adapting their game to the current networking system. But the upside was that they were able to take this feedback to the networking team who could use it to inform future decisions regarding networking in Unity. We got this feedback because the conventions of a particular game genre inspired us to use our game engine in a certain way. And we got it even though networking was not part of the coverage scoped for the scenario. Automation or manual test-cases will not give you this kind of information.

这是场景测试所产生的各种反馈的示例。 场景之一是创建一个实时策略游戏(下面的屏幕截图)。 它旨在获得对AI，粒子和地形的覆盖。 但是，一旦游戏逐渐成形，自然就会想到增加多人游戏的想法。 很快，团队就遇到了使他们的游戏适应当前网络系统的问题。 但是好处是，他们能够将这些反馈信息反馈给网络团队，他们可以利用该反馈为Unity中有关网络的未来决策提供依据。 我们之所以得到此反馈，是因为特定游戏类型的惯例启发了我们以某种方式使用我们的游戏引擎。 即使网络不属于该方案涵盖的范围，我们也能做到。 自动化或手动测试用例不会为您提供此类信息。

We are continuously looking for ways to improve how we do Scenario Testing. One thing we are looking at is continuing work on some of the scenarios across multiple Scenario Test Weeks (i.e. multiple versions of Unity). Until now, we started over with new game productions each test-phase. We hope that this can give us a better approximation of both upgrading Unity during a production and also working on a project for longer than hacking the first prototype together.

我们一直在寻找改善场景测试方法的方法。 我们正在考虑的一件事是在多个方案测试周(即Unity的多个版本)中的某些方案上继续工作。 到目前为止，我们在每个测试阶段都从新游戏产品开始。 我们希望这可以使我们更好地近似于在生产过程中升级Unity以及将项目工作比将第一个原型一起**更长的时间。

We don’t presume that our little scenarios can accurately replicate the experience of working with Unity on a real project. Still, we feel that we can make a pretty good approximation Scenario Testing. And – not to forget – it’s actually a fun way to test Unity.

我们不认为我们的小场景可以准确地复制在实际项目中使用Unity的经验。 尽管如此，我们仍然认为我们可以做一个很好的近似方案测试。 而且-勿忘-这实际上是测试Unity的一种有趣方式。

Here are a few small samples of projects that were created during Scenario Test Week for Unity 5.5. Note that in addition to the coverage mentioned for each project, all projects also used Unity Collaborate and Unity Cloud Build. We dog-food our services offerings as well. Asset Store is also tremendously helpful as it allows teams to use high-quality assets in their projects. Not only does this boost the production value of the games, it also contributes to testing by allowing more complex and realistic data/assets to be used.

以下是在Unity 5.5的“方案测试周”中创建的一些项目小样本。 请注意，除了每个项目提到的范围外，所有项目还使用 Unity Collaborate 和 Unity Cloud Build 。 我们也为我们的服务提供狗粮。 Asset Store也非常有用，因为它允许团队在项目中使用高质量的资产。 这不仅可以提高游戏的生产价值，还可以通过允许使用更复杂，更真实的数据/资产来帮助测试。

FPS game Team size: 4 people. Target platform: Desktop. Focus areas: Audio, General Graphics, Substances, Post-processing

FPS游戏 团队人数：4人。 目标平台：台式机。 重点领域：音频，通用图形，物质，后处理

3D RTS Team size: 4 people. Target platform: Desktop. Focus areas: AI, Particles, Terrain/SpeedTree integration

3D RTS 团队规模：4人。 目标平台：台式机。 重点领域：AI，粒子，地形/ SpeedTree集成

2D JRPG Team size: 3 people. Target platform: Android. Focus areas: 2D, UI, Particles

2D JRPG 团队人数：3人。 目标平台：Android。 重点领域：2D，UI，粒子

2D Platformer Team size: 3 people. Target platforms: iOS, TVOS, PS Vita. Focus areas: 2D, Animation 2D, Physics 2D

2D Platformer 团队规模：3人。 目标平台：iOS，TVOS，PS Vita。 重点领域：2D，动画2D，物理2D

VR Museum Team size: 2 people. Target platform: VR. Focus areas: Graphics – general, Lighting, Post Processing

VR博物馆 团队人数：2人。 目标平台：VR。 重点领域：图形–常规，照明，后期处理

Slender Man clone Team size: 3 people. Target platforms: PS 4, VR. Focus areas: Graphics – general, Terrain/SpeedTree integration, Substance integration

苗条的男人克隆 团队人数：3人。 目标平台：PS 4，VR。 重点领域：图形–常规，地形/ SpeedTree集成，物质集成

翻译自: https://blogs.unity3d.com/2017/01/03/a-look-inside-scenario-testing-at-unity/