4. 非功能性测试的若干问题
· One key issue is when to do NFT? The problem with non-functional defects is that most of them will be related to the design and fixing them will need a fix in the design and therefore effort in fixing may be high. This warrants that NFT betaken up as early in the test cycle as possible or the fix may adversely affect meeting the delivery schedules. Another reason for advocating an early execution of NFT is that some of the NFT defects like usability issues maybe perceived to be of a lower priority and the probability of such defects being left open as limitations is quite high. The project management tends to postpone these since changing the code may potentially affect even the working functionality.
一个关键的问题是何时进行非功能性测试。大部分非功能性的缺陷与设计有关，故要修复它们就需要修改设计，因此修复代价是很高的。这就说明非功能性测试需要在测试周期中尽可能早的进行，否则修复活动将反过来影响到提交进度的实现。提倡尽早实施非功能性测试的另一个原因是：类似可用性这样的非功能性属性在测试中出现的缺陷可能被认为是低优先级的，并且这种缺陷成为一个公开限制的可能性非常高。项目管理者倾向于把这些缺陷的修复推后，因为改变代码可能对系统功能潜在地产生影响。
But the problem with scheduling NFT early, is that in many cases the system need to attain a minimum level of stability and functionality completeness for the test cases to run. Examples are load, performance and configuration testing. This will mean that the system functionality testing has to be done and any defects found from this need to be fixed before NFT can be carried out. If the system is not robust and stable enough before performance or load testing is attempted, the test team maybe wasting a lot of effort in restarting the system etc..
然而，把非功能性测试安排到早期也存在问题。那就是在很多情况下系统需要获得一个最低程度的稳定性和功能完整性来运行测试用例。例如载入测试、性能测试和配置测试。这就意味着在非功能性测试完成之前，系统功能性测试应该已经完成并且从中找到的任何缺陷都应该得到修复。假如在性能测试和载入测试进行之前系统达不到足够的健壮和稳定的话，测试团队很可能在重起系统等方面等待很长时间。
Considering the above mentioned factors, the following approach is suggested:
考虑到以上因素，这里提出如下建议：
Initial Rounds : Functionality testing, Usability testing, Security testing
第一轮：功能性测试、可用性测试、安全性测试。
Intermediate Rounds : Functionality (Remaining and regression), Configuration Testing, Recovery testing
中间轮：功能性测试(剩余的和回归的)、配置性测试、恢复性测试。
Final rounds : Volume, Stress and performance
This is a general guideline for sequencing and will need to be tailored for individual projects.
这只是一个对测试顺序的大概说明，在具体项目中还可以自行取舍。
If reliability assessment is done through analysis, it will continue throughout test execution.
如果已经通过分析完成了可靠性评估，那么还需要通过继续测试来完成最终的评估。
· There are a few issues related to regression testing of NF requirements to be considered. 100% regression is neither practical nor advisable. Examples of issues are
还有几点是关于对非功能性需求的回归测试的，100%的回归测试既不现实也不被提倡。以下是两个例子：
– Repetition of time consuming volume testing can affect release schedule.
– It will not be possible to get the desired feedback if the same subject is used again for doing usability testing of same procedures
— 容量测试中大量时间的消耗会影响项目的进度
— 如果在可用性测试中对于同样的过程使用同样的主题，是不可能得到想要的反馈的。
Therefore normally only defect fixes are verified for most cases unless there are major changes in implementation, UI design etc.. Exceptions can be performance and stress testing.
因此，除非在实现，用户接口设计等方面有重大变化，通常情况下仅仅校验缺陷的修复。而性能测试和压力测试又属例外情况。
5. Performance Testing
5. 性能测试
The importance of performance of the system to the customer is somewhat better understood than other NF requirements. Testing for the stated requirement also is performed in many projects. But this testing is sometimes limited to the measurement of one value, say, throughput rate or response time. But this need to be done under differing processing and configuration conditions.
对于客户来说，系统性能的重要性比其他非功能性需求要好理解一些。但在很多项目中，对规定需求的测试还是要进行。然而，测试往往局限于对某个值的测量，比如吞吐率或者响应时间。尽管如此，在不同的处理和配置条件下，这些测试还是要去实施。
The performance of the system depends on various factors including configuration of the system, external factors and current state of the system. Therefore, test team should take into account these while doing the performance benchmarking [6]. Configuration of the system includes all hardware and software configurations like number of processors, RAM, DB server software used etc.. External factors may include things like network load and switching system used. Internal factors may include transactions processed since startup, current load on the system, any failure conditions etc.. The various steps in performance testing are listed below:
系统的性能取决于很多因素，这些因素包括系统的配置，外部条件以及系统的当前状态。因此，当测试团队在做性能基准的时候需要考虑这些因素。系统配置包含所有的硬件和软件，例如处理器数量、RAM、数据库服务器软件等等。外部因素包含网络负载和交换机等。内部因素包含启动后的业务处理、系统当前负载以及任何失败条件等等。以下列出了性能测试有关的若干步骤：
1) Identify the parameters to be measured
Examples are call setup time, TPS etc. This may be taken from requirements book.
1) 确定要测量的参数。例如调用启动时间，TPS等等。这些可以从需求说明书中得到。
2) Identify the different external factors, system configuration parameters and internal factors which affect performance. It will be a good idea to brainstorm and come up with a fish-bone diagram.
2) 确定不同的外部条件、系统配置参数以及能够影响性能的内部因素。使用头脑风暴法以及鱼骨图会是个不错的主意。
3) Identify discrete values each of these parameters can take and shortlist the factors and values for which you plan to carry out the testing. It is essentially identifying interesting profiles for testing.
3) 确定离散值，使每一个参数都能反映出影响性能的因素并且可以对这些值可以进行测试。确定那些引起兴趣的方面并进行测试很重要。
4) Identify a default, most common value for each factor.
4) 为每一个因素确定一个默认的，最常用的值。
5) Identify the measurement method
This may require tools, special build of the product, special setup etc.. Plan and implement these
5) 确定测量方法。这可能需要使用工具、产品的特殊构建、特殊的启动等等。规划并实现它们。
6) Do the measurement varying one parameter at a time keeping all the others at their default values.
6) 在保持其他所有默认值不变的情况下，每次测量的时候只改变一个参数。
7) Plot the results so that the effect of each parameter on performance is identified.
7) 对测试结果进行划分，这样可以确定每一个参数对性能的影响程度。
Steps 1-4 are done during test planning phase, step 5 is done as part of test requirements, step 6 relates to test execution and step 7 is part of test reporting. The advantage of this is that marketing or customer support has data to advise on the ideal configuration for a customer based on his performance and other requirements.
步骤1到4在测试计划阶段实施，步骤5作为测试需求的一部分，步骤6与测试执行有关而步骤7是测试报告的一部分。这样做的好处是可以基于性能和需求来为市场部或者客户支持部提供数据，以便让他们为客户提供理想的配置。
Performance testing can be accomplished in parallel with Volume and stress testing because you want to assess performance under all load conditions. Volume testing is performed to find weaknesses of the system with respect to its handling large amounts of data over time. Stress testing is carried out to find out the system behavior when subjected to large numbers of transactions during peak period.
性能测试可以和容量测试以及压力测试并行地完成，这是因为你想在所有的载入条件下都到达性能指标。容量测试通过系统对大量数据的处理时间来找到系统的弱点。而压力测试则是通过系统在高峰期的大量的业务处理来找出系统的缺陷。
If the performance does not measure up to the requirements / expectations, profiling tools are used to find the performance bottle necks in the system. Test team will need to assist development team in this as well.
如果系统性能不能满足需求的期望值，那么就要使用剖析工具来找出制约系统性能的瓶颈。测试团队此时同样要协助开发团队。
Other activities normally associated with performance testing are resource utilization measurements. This normally includes memory and CPU utilization and the testing is carried out under volume and stress conditions.
其他与性能测试协同进行的过动是资源利用测量。它通常包含内存和CPU的利用率并且是在容量和压力的条件下进行测试。
6. Usability Testing
文章来源于领测软件测试网 https://www.ltesting.net/
