In this project we propose a new deep multibranch neural network to solve the tasks of artist, style, and genre categorization in a multitask formulation. In order to gather clues from low-level texture details and, at the same time, exploit the coarse layout of the painting, we propose a deep neural network composed of multiple branches fed with crops at different resolutions. We propose and compare two different crop strategies: the first one is a random-crop strategy that permits to manage the tradeoff between accuracy and speed; the second one is a smart extractor based on Spatial Transformer Networks trained to extract the most representative subregions. Furthermore, inspired by the results obtained in other domains, we experiment the joint use of hand-crafted features directly computed on the input images along with neural ones.

Online Demo Available!

Try out our online demo: (link)

MultitaskPainting100k Dataset

The dataset used for the evaluation of our multitask deep multibranch neural network has been obtained from the Painter by Numbers Kaggle competition (link). However the original split is not suitable for our task. To accomplish our task we select a subset of the original dataset such that there are at least 10 images in every class for a total of 1508 artists, 125 styles and 41 genres. We call this selection the MultitaskPainting100k dataset. The dataset is split in two parts: a random 70% belonging to the train set and the remaining 30% to the test set.

Paintings from the MultitaskPaintings100k dataset. Each row contains samples from a different artist. For each artist we show paintings with different genres and styles. Color coding is used to distinguish between genres and styles.

Distributions of number of samples available for each of the 1508 artists, 41 genres and 125 styles within theMultitaskPaintings100k dataset. The names of classes are partially shown for lack of space.

Link to download the dataset.

@article{bianco2019-multitask,
 author = {Bianco, Simone and Mazzini, Davide and Napoletano, Paolo and Schettini, Raimondo},
 year = {2019},
 pages = {90-101},
 title = {Multitask Painting Categorization by Deep Multibranch Neural Network},
 volume = {135},
 journal = {Expert Systems with Applications},
 pdf = {/download/Multitask_painting_categorization_by_deep_multibranch_neural_network.pdf},
 doi = {10.1016/j.eswa.2019.05.036},
 issn = {0957-4174},
 projectref = {http://www.ivl.disco.unimib.it/activities/paintings/}}

@inproceedings{bianco2017deepMultibranch,
 author = {Bianco, Simone and Mazzini, Davide and Schettini, Raimondo},
 year = {2017},
 pages = {414-423},
 title = {Deep Multibranch Neural Network for Painting Categorization},
 volume = {10484},
 publisher = {Springer},
 series = {Lecture Notes in Computer Science},
 booktitle = {Image Analysis and Processing - ICIAP 2017},
 doi = {10.1007/978-3-319-68560-1_37}}