Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Most A/B tests should not be done in a production like way. Grab some post-it notes and and sketch out both A and B: then watch users work with it.

For a lot of what you want to know the above will give better information than a 100% polished A/B test in production. When people see a polished product they won't give you the same type of feedback as they will for an obvious quick sketch. The UI industry has gone wrong by making A/B in production too easy, even though the above has been known for many decades.

(A/B in production isn't all bad, but it is the last step, and often not worth it)



You are pushing for bit of too much of name-misuse. A/B tests are something what the entire point is that you run them on the real world, and gather how real users react.

Design-time user testing has been a thing for much longer than A/B tests. They are a different thing.

I mean, your point stands. But you can't do A/B tests on anything that is not production, those your are recommending are a different kind of tests.


I'll accept your definition correction. However I think my point still stands: there are better things than A/B testing to get the information you need.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: