Fractals can help AI learn to see more clearly—or at least fairly

The trouble is, assembling a dataset like ImageNet by hand takes a lot of time and effort. The images are typically labelled by low-paid crowdworkers. Datasets might also contain sexist or racist labels that can bias a model in hidden ways as well as images of people who have been included without their consent. There’s evidence these biases can creep in even in pretraining.

However, fractal patterns can be found in everything from trees and flowers to clouds and waves. This made the team at Japan’s National Institute of Advanced Industrial Science and Technology (AIST), the Tokyo Institute of Technology and Tokyo Denki University wonder if these patterns could be used to teach an image-recognition system the basics of image recognition, instead of using photos of real objects.

The researchers created FractalDB, an endless number of computer-generated fractals. Some look like leaves, others look like snowflakes or snail shells. Each group of similar patterns was automatically given a label. They then used FractalDB to pretrain a convolutional neural network, a type of deep-learning model commonly used in image recognition systems, before completing its training with a set of actual images. They found that it performed almost as well as models trained on state-of-the-art datasets, including ImageNet and Places, which contains 2.5 million images of outdoor scenes. 

Does it work? Anh Nguyen at Auburn University in Alabama, who wasn’t involved in the study, isn’t convinced that FractalDB is a match for the likes of ImageNet. He has studied how abstract patterns can confuse image recognition systems. “There is a connection between this work and examples that fool machines,” he says. He would like to explore how this new approach works in more detail. But the Japanese researchers think that by tweaking their approach, computer-generated datasets like FractalDB could replace existing ones.  

Why fractals: The researchers also tried training their AI using other abstract images, including ones produced using Perlin noise, which creates speckled patterns, and Bezier curves, a type of curve used in computer graphics. But fractals gave the best results. “Fractal geometry exists in the background knowledge of the world,” says lead author Hirokatsu Kataoka at AIST.