> scanning for problems and clicking an "approve button", that isn't a lot of work though.
But you're actually mentally listing requirements for each one of those snapshots you check, which hopefully is the same as the previous person who run it, but who knows?
> Yes they will because the initial conditions also had a snapshot attached. If your snapshot testing is even mildly fancy it will come with a diff too.
Maybe I didn't explain properly. Say I create a component, and use snapshot testing to verify that "This is how it should look". Now next person changes something that makes that snapshot "old", and the person needs to look at the diff and new component, and say "Yeah, this is now how it should look". But there is a lot of things that are implicitly correct in that situation, instead of explicitly correct. How can we be sure the next person is mentally checking the same requirements as I did?
> If you made a change and you can see the previous snapshot, current snapshot and a diff and you never know if the change was ok then you probably shouldn't be working on the project in the first place.
It seems to work fine for very small and obvious things, but for people make changes that affect a larger part of the codebase (which happens from time to time if you're multiple people working on a big codebase), it's hard to needing to implicitly understand what's correct everywhere. That's why unit/functional tests are so helpful, they're telling us what results we should expect, explicitly.
> I've seen hundreds of unit tests that [...] With a user facing snapshot it's obvious.
I agree that people generally don't treat test code with as much thought as other "production" code, which is a shame I suppose. I guess we need to compare "well done snapshot testing" with "well done unit/functional testing" for it to be a fair comparison.
For that last part, I guess we're just gonna have to agree to disagree, most snapshot test cases I come across aren't obvious at all.
>Maybe I didn't explain properly. Say I create a component, and use snapshot testing to verify that "This is how it should look". Now next person changes something that makes that snapshot "old", and the person needs to look at the diff and new component, and say "Yeah, this is now how it should look". But there is a lot of things that are implicitly correct in that situation, instead of explicitly correct. How can we be sure the next person is mentally checking the same requirements as I did?
This is a problem that applies equally to code and all other tests to an even greater extent. For them it is a problem dealt with with code reviews.
The great thing about snapshots which doesn't apply to code and functional tests is that they can be reviewed by UX and the PM as well. In this respect they are actually more valuable - PM and UX can spot issues in the PR. The PM doesn't spot when you made a mistake interpreting the requirements that is only visible in the code of the functional test or when it's something the functional test didn't check.
>It seems to work fine for very small and obvious things, but for people make changes that affect a larger part of the codebase (which happens from time to time if you're multiple people working on a big codebase), it's hard to needing to implicitly understand what's correct everywhere
It should not be hard to ascertain if what you see is what should be expected. E.g. if text disappears from a window where there was text before and you only made a styling change then that's obviously a bug.
>I guess we need to compare "well done snapshot testing" with "well done unit/functional testing"
Snapshots are not a replacement for functional tests they are a part of good functional tests. You can write a functional test that logs in and cheaply checks some arbitrary quality of a dashboard (e.g. the div it goes in exists) or you can write a functional test that cheaply snapshots the dashboard.
The latter functional test can give confidence that nothing broke when you refactor the code underneath it and the snapshot hasn't changed. The former can give you confidence that there is still a div present where the dashboard was before. This is significantly less useful.
But you're actually mentally listing requirements for each one of those snapshots you check, which hopefully is the same as the previous person who run it, but who knows?
> Yes they will because the initial conditions also had a snapshot attached. If your snapshot testing is even mildly fancy it will come with a diff too.
Maybe I didn't explain properly. Say I create a component, and use snapshot testing to verify that "This is how it should look". Now next person changes something that makes that snapshot "old", and the person needs to look at the diff and new component, and say "Yeah, this is now how it should look". But there is a lot of things that are implicitly correct in that situation, instead of explicitly correct. How can we be sure the next person is mentally checking the same requirements as I did?
> If you made a change and you can see the previous snapshot, current snapshot and a diff and you never know if the change was ok then you probably shouldn't be working on the project in the first place.
It seems to work fine for very small and obvious things, but for people make changes that affect a larger part of the codebase (which happens from time to time if you're multiple people working on a big codebase), it's hard to needing to implicitly understand what's correct everywhere. That's why unit/functional tests are so helpful, they're telling us what results we should expect, explicitly.
> I've seen hundreds of unit tests that [...] With a user facing snapshot it's obvious.
I agree that people generally don't treat test code with as much thought as other "production" code, which is a shame I suppose. I guess we need to compare "well done snapshot testing" with "well done unit/functional testing" for it to be a fair comparison.
For that last part, I guess we're just gonna have to agree to disagree, most snapshot test cases I come across aren't obvious at all.