I also wonder how many of the numerous AI proponents in HN comments are subject to the same effect. Unless they are truly measuring their own performance, is AI really making them more productive?
You could go the same way as the study, flip a coin to use AI or not, write down the task you just did, the time you thought the task took you and the actual clock time. Repeat and self-evaluate.
Sample of 16 is plenty if the effect is big enough.
It’s also not a sample size of 1, it’s a sample size of however many tasks you do because you don’t care about measuring the effect AI has on anyone but yourself if you’re trying to discern how it impacts you.
It's the most representative sample size if you're interested in your own performance though. I really don't care if other people are more productive with AI, if I'm the outlier that's not then I'd want to know.
