Rorschach2Txt

The Rorschach test was developed in 1921 in order to measure disorder through analysing what a viewer sees in a seemingly ambiguous image.

We were interested in seeing what a neural network would see in a Rorschach test. We used im2txt (image to text), which is a neural network specifically trained to look at images, and generate a short description of what it thinks it sees. What we found was the descriptions that it showed highlighted the limitations of the neural networks we were using, and the great influence that the training set has on the outputs. Which is to say that either the neural network has some Freudian attachments to Nintendo WiiMotes and toothbrushes, or that the scope of the training data so limited that it literally couldn’t give any other description.

The experiments started with feeding ambiguous music videos (like the ones by OK-GO) into the system, and watching it try understand what it saw. What became more interesting was the team working to understand what the neural network saw. It became a guessing game. Why did it just see a toothbrush? Was it the checkered background that looks like bathroom tiles? Was it the person in front of a mirror?

Eventually Wekinator was trained to recognise our facial expressions, using that to generate a Rorschach test that we could then feed into im2txt.