I finished the illustration set and started on the photographic side, and somewhere in the middle I realised I cannot run this test the way I planned. The reason is simple, and it took me a while to see it clearly. The illustrations and the photographs are not at the same level of quality, and because of that, any result I got would be about quality, not about the medium.
That is the whole problem in one sentence. I set out to compare illustration and photography. But if the two sets do not match in quality, then the thing people actually react to is which one looks better made, not whether it is drawn or photographed. The comparison quietly stops being about what I wanted it to be about.
And the mismatch is not in one direction. It is not that the illustrations are good and the photographs are bad, or the other way around. Each set is uneven in its own way. The illustrations are very consistent but clearly stylised. The photographs look real but shift from image to image and never quite hold together as one thing. So I cannot even say one medium is winning. They are simply unequal in different ways, and that is enough to break the test.
This is where I have to be honest about quality itself. I had been treating it as a background detail, something I would clean up later. But it is not a detail. Quality is one of the variables in this experiment, maybe the most important one, and I had not been controlling it at all. As long as it floats freely, it sits on top of everything else people see, and it drowns out the difference I am trying to measure.
So the real question is not finished, it is just beginning. If quality is a variable, then I have to find a way to hold it steady before I can fairly compare anything. Both versions would need to sit at a similar level of finish, so that the only thing left to react to is illustration versus photography. Right now I do not know how to guarantee that, and I would rather admit it than run a test I already know I cannot defend.
I do not have the answer yet. What I have is a clearer question. Before I can test what each medium does, I have to figure out how to make the two sides equal enough that the test is actually fair. That is the problem the next block has to deal with.
