Talk:Load testing

The link to Grinder is wrong. It should be http://grinder.sourceforge.net/ —Preceding unsigned comment added by ToddCostella (talk • contribs) 17:40, 22 January 2010 (UTC)

]]]]]==VUser== What is a VUser? As someone who knows little about this topic, a sentence or two on its definition would be helpful. Ssnseawolf (talk) 15:39, 25 February 2009 (UTC)

helfulTool

What is the criteria for listing vendors on the tools section? Marketshare? Annual sales? Popularity? Czei 14:41, 13 March 2007 (UTC)

What is the criteria for being listed in the tools section? The first document, for example, is not about the load testing process, but is a software manual, while an article about how to do load testing on .NET applications is removed as "spam". Czei 14:43, 13 March 2007 (UTC)

The two pages should be merged since the separate "tools" section can't possibly work given the guidelines: "Promotional articles about yourself, your friends, your company or products, or articles created as part of a marketing or promotional campaign, will be deleted in accordance with our deletion policies." Any listing of "tools" would have to include products, and therefore would be categorized as "marketing". To be fair, any listing of "tools" which have to include all tools or none, otherwise you'd have to get into the criteria why some are listed others are not. (Czei 14:41, 15 June 2007 (UTC)) . —Preceding unsigned comment added by 121.243.192.105 (talk) 06:15, 21 November 2007 (UTC)

Software Load Testing is Misleading

The world of software load testing is changing and this article (as well as the list of load testing tools) is out of date, incomplete, and misleading. Disclaimer: I'm the founder of a new startup, BrowserMob, and I have a very clear conflict of interest and therefore cannot make edits to any of these pages.

This article on claims that software load testing shares very little similarity with functional testing tools, primarily because load testing operates on the protocol level whereas functional testing tools work at the GUI level. I have several concerns with this:

• This paragraph would much more informational if it explained why most load testing tools work at the protocol level. Ex: because GUI-level testing for even 1 VU consumes much more memory that is practical to use for large scale load testing.
• This is no longer true. In November 2008, I launched BrowserMob, which provides load testing via Selenium_(software), a functional testing tool. It uses Selenium for both record and playback. Therefore, this is a load testing tool which does not operate at the protocol level and instead does indeed work just like a functional testing tool.
• The technique of protocol level playback is becoming more of a problem as web applications evolve and become more complex. I think this topic needs to be discussed somewhere. I recently published an article on http://ajaxian.com that discusses why load testing Ajax applications is hard.

BrowserMob works by leveraging cloud computing, which makes it affordable to actually use real browsers for GUI-based interaction in a load test. This technique has also been referenced independently by people on the Amazon EC2 forums.

I also agree that the load testing tools section seems a bit odd. Some of the tools referenced are just not accurate (HtmlUnit and Selenium are almost never used for load testing, except in the case of Selenium with BrowserMob). Also, some commercial tools are referenced while some are not, which seems to violate the neutrality of Wikipedia.

I think a better approach would be to eliminate the category page and roll it up in to this article, which a more in-depth discussion of the different approaches and techniques. I would also remove HtmlUnit and Selenium entirely and put them in the proper category. I would love to do this myself (I'm the founder of OpenQA and SeleniumHQ, so I have the expertise) but I clearly have a conflict of interest.

Any thoughts would be appreciated! Plightbo (talk) 18:17, 3 January 2009 (UTC)

Market Share

A comment left by a recent editor reads: "none of these tools have 5% market share (unless you only include Fortune 500)". I would argue that a tools survey, such as the one done by Better Software Magazine and StickyMinds.com is sufficient for determining market share. --Walter Görlitz (talk) 01:12, 16 January 2010 (UTC)

Poor Criteria

It is interesting to note that this article does not even mention Apache Bench, yet Apache Bench is distributed with every instance of Apache Web Server and I would contend that gives them at least 60%. Arbitrary and Capricious seems to be the watch word for this article. — Preceding unsigned comment added by AreYouFreakingKidding (talk • contribs) 17:46, 6 December 2012 (UTC)

Feel free to provide a reference to support that statement. Use of JMeter is common while use of ApacheBench isn't. The tools surveys that I have seen from Better Software and other locations support that. While it may be distributed, it whether it's used or not that counts for market share. We don't just add thinks because someone thinks they should be added. --Walter Görlitz (talk) 18:00, 6 December 2012 (UTC)

Computational Load Testing

I think there should be something that ties this subject into the computer world with computer stress tests and servers/workstations. It would be usefull to try to link to the CPU or computer system stress test page. This gives details and allows for more central location of the computer aspect of load testing. If others agree that it is relevant we can promote it to the main page.

There is a clear difference between load testing and steress testing. Load testing tests a system for behavior at normal and peak loads. The expectation is to ensure it behaves within an acceptable range established by requirements. Stress tests on the other hand test a system until it fails to ensure the system degrades and recovers gracefully. — Preceding unsigned comment added by 149.101.1.120 (talk) 14:26, 12 February 2014 (UTC)

There is no unified nomenclature. What one glossary, certifying body or company calls load testing could be called stress testing or performance testing or something else in a different glossary, certifying body or company. Walter Görlitz (talk) 15:22, 12 February 2014 (UTC)

Concurrency testing

Is concurrency testing another synonym for load testing? Can anyone verify this? -- AnonymousDDoS (talk) 02:53, 26 June 2012 (UTC)

Not really. Usually when testing is done, it is done by one or at most a few individuals and this would be on larger systems and on different parts of the system. Concurrency testing is when multiple people are attempt to access the same part of the system at the same time. Perhaps one is writing a record while another tries to read it. The following article has a short section describing it: http://www.stickyminds.com/s.asp?F=S6920_ART_2 . Whether it may incorrectly be used synonymously with the term load testing isn't really clear. --Walter Görlitz (talk) 04:08, 26 June 2012 (UTC)

Perhaps we could add a section to load testing, or create a totally new article and provide a red link? -- AnonymousDDoS (talk) 13:58, 26 June 2012 (UTC)

This Article seems to be incredibly biased

The listing of load testing tools on this page seems to be arbitrary and capricious. Two of the most popular and readily available tools aren't even listed here. Why is one person granted the power to decide what does and does not go on these pages. I thought this was a community project. — Preceding unsigned comment added by AreYouFreakingKidding (talk • contribs) 17:40, 6 December 2012 (UTC)

I totally agree with this. — Preceding unsigned comment added by 50.193.56.110 (talk) 00:41, 13 February 2013 (UTC)

Neither arbitrary nor capricious. The basis for inclusion is common on Wikipedia lists: notability. If there is an article, it may be included here. Not all lists follow this guideline, but many do. If it were not enforced we would have an inordinately long list of small projects that aren't useful for the broad public. This isn't a stand-alone list either since that is simply a short section listing the tools. All of the most popular and readily available tools, according to the tools surveys I have seen, are listed here. Which tools do you think are missing? Walter Görlitz (talk) 14:32, 2 May 2013 (UTC)

Wrk, Tsung, k6, Eggplant Performance, and Artillery would definitely be a good start. This list is currently incredibly outdated and every attempt to rectify that seems to lead to an instant revert. 217.210.164.244 (talk) 16:25, 1 April 2021 (UTC)

Any attempt to add a product that has no article to the table should be reverted. There's no reason that they cannot be added, with WP:SECONDARY sources in the prose though. Walter Görlitz (talk) 00:04, 2 April 2021 (UTC)

Undid revision 1083961456 @Walter Görlitz:

With SoapUI you can perform load tests . LeFelix (talk) 08:41, 22 April 2022 (UTC)

Thanks. I had not seen that feature before, but I haven't used it in about 12 years. Thanks again. Would you mind updating the SoapUI article with that detail as well? Walter Görlitz (talk) 15:27, 22 April 2022 (UTC)

No problem and sorry for the late reply. As you have already seen, I only added a link to the SoapUI article, since it was already noted that this tool can run load tests. I will add SoapUI to the list again. :) LeFelix (talk) 15:37, 5 May 2022 (UTC)

References