Deepstyle
Neural Art
Want to buy my Deepstyle prints? Check out Redbubble.
Don’t know what “Deepstyle Neural Art” is?
This is based on the research paper by Leon A. Gatys, Alexander S. Ecker and Matthias Bethge, you can read about their findings here.
In short, you can apply styles from famous painters like Vincent van Gogh, Rembrandt, Edvard Munch or whomever you choose to your photos for a interesting effect.
As a photographer i find this interesting, since i have a large supply of my own material to apply this to.
To get this to work you need a computer with Ubuntu 14.04 LTS or Ubuntu 16.04 LTS installed, or you could rent a box in the cloud from Digitalocean.
Other useful software that you might want to have handy on your computer to transfer the output files from the Ubuntu box would be PuTTY (for issueing commands in SSH) and WinSCP (for file transfer).
I hope you find this tutorial useful!
Assuming you have Ubuntu installed, lets get to it!
UBUNTU 16.04 LTS:
Do not do this as root, first create a user.
Example (if you are logged in as root) :
sudo adduser neural
sudo usermod -aG sudo neural
su neural
cd ~
Dependencies needed
(copy/paste commands below to your terminal):
- Make sure everything is up to date
sudo apt-get update && sudo apt-get upgrade
- Install GIT
sudo apt-get install git
- Install OpenBLAS
sudo apt-get install libopenblas-dev
- Install Torch
git clone https://github.com/torch/distro.git ~/torch --recursive
cd ~/torch; bash install-deps;
./install.sh
Answer YES when install is finished and do a :source ~/.bashrc
- Update LuaRocks needed for this
luarocks install torch
luarocks install nn
luarocks install image
luarocks install lua-cjson
Installing Fast Neural Style:
We will be using Fast Neural Style from Justin Johnson on Github.
- Make sure you are in home directory
cd ~
- Clone Neural Style repo
git clone https://github.com/jcjohnson/fast-neural-style.git
- Go into directory
cd fast-neural-style
- Download models (around 655MB download)
bash models/download_style_transfer_models.sh
bash models/download_vgg16.sh
Keep in mind that you might have to edit the scripts files on change http to https for the downloads to work. - To do this:
sudo nano models/download_style_transfer_models.sh
sudo nano models/download_vgg16.sh
After this you should be all setup for churning out neural art.
To get started:
- Make sure you are in correct directory
cd ~/fast-neural-style
- Command for running Neural Style (without GPU)
th slow_neural_style.lua -content_image images/content/YOURCONTENTFILE.jpg -style_image images/styles/YOURSTYLEFILE.jpg -output_image YOURFILENAME.png -num_iterations 1000 -print_every 1
This will output a image 512px wide, which is the default setting, and print every iteration on screen with a total of 1000 iterations (this can be changed your liking)
Check Fast Neural Style on Github for more information on parameters.
You can change this by doing:
th slow_neural_style.lua -style_image NAMEOFPAINTING.jpg
-content_image YOURPHOTO.jpg -gpu -1 -image_size SIZEINPIXELS
-output_image NAMEOFOUTPUT.png -num_iterations NUMBEROFITERATIONS -print_every ITERATIONSTOPRINTONSCREEN
Be advised that the pixel size will affect how much RAM is used, for a 512px image you should have a minimum of 4GB RAM.
Using a box in the cloud can help you with this, like a compute optimized server with 64GB RAM and 32 dedicated hyperthreads from Digitalocean.
I have been making neural art with pixel sizes up towards 2560px on Digitalocean with their largest servers for compute/ml setups.
- Make sure you are in correct directory
UBUNTU 14.04 LTS:
Dependencies needed
(copy/paste commands below to your terminal):
- Make sure everything is up to date
sudo apt-get update && sudo apt-get upgrade
- Install GIT
sudo apt-get install git
- Install LUA
sudo apt-get install lua5.2
- Install LuaRocks
sudo apt-get install luarocks
- Install LuaJIT
sudo apt-get install luajit.
- Install Torch
sudo curl -s https://raw.githubusercontent.com/torch/ezinstall/master/install-all | bash
- Install LoadCaffe
sudo apt-get install libprotobuf-dev protobuf-compiler
sudo luarocks install loadcaffe
- Update LuaRocks needed for this
sudo luarocks install paths
sudo luarocks install sys
sudo luarocks install image
sudo luarocks install nn
sudo luarocks install optim
Installing Neural Style:
We will be using Neural Style from Justin Johnson on Github.
- Make sure you are in home directory
cd
orcd ~
- Make directory
mkdir DeepStyle
- Go into directory
cd Deepstyle
- Clone Neural Style repo
sudo git clone https://github.com/jcjohnson/neural-style.git
- Go into directory
cd neural-style
- Download models (around 655MB download)
sudo sh models/download_models.sh
After this you should be all setup for churning out neural art.
To get started:
- Make sure you are in correct directory
cd ~/DeepStyle/neural-style
- Command for running Neural Style (without GPU)
sudo th neural_style.lua -style_image NAMEOFPAINTING.jpg -content_image YOURPHOTO.jpg -gpu -1 -output_image NAMEOFOUTPUT.png
This will output a image 512px wide, which is the default setting.
Check Neural Style on Github for more information on parameters.
You can change this by doing:
sudo th neural_style.lua -style_image NAMEOFPAINTING.jpg
-content_image YOURPHOTO.jpg -gpu -1 -image_size SIZEINPIXELS
-output_image NAMEOFOUTPUT.png
Be advised that the pixel size will affect how much RAM is used, for a 512px image you should have a minimum of 4GB RAM.
Using a box in the cloud can help you with this, like Digitalocean.
Some of my results:
THE ART OF PATIENCE:
Generating these images can take time, depending on your hardware, with the use of GPU it will be a lot faster, or by using a cloud service like Digitalocean (non GPU, but quite affordable) or Amazon EC2 (for GPU cloud)
If you’re going with the Amazon EC2 (g2.2xlarge or g2.8xlarge) alternative there is a good tutorial on Reddit for this.
If you decide to use the GPU on your Ubuntu box here’s how to do it:
(requires a CUDA capable card from Nvidia)
- Go to home directory
cd
orcd ~
- Download repo from Nvidia
sudo wget http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1404/x86_64/cuda-repo-ubuntu1404_7.5-18_amd64.deb
- Install repo
sudo dpkg -i cuda-repo-ubuntu1404_7.5-18_amd64.deb
- Update packagelist
sudo apt-get update
- Install CUDA
sudo apt-get install cuda
- Reboot to load driver
sudo reboot
- Check if card is being recognized
nvidia-smi
- Install CUDA backend for torch
sudo luarocks install cutorch
sudo luarocks install cunn
- Check if backend is working
cd ~/DeepStyle/neural-style
sudo th -e "require 'cutorch'; require 'cunn'; print(cutorch)"
This should produce a output. - Test with neural style
sudo th neural_style.lua -gpu 0 -print_iter 2
USES:
There are many possibilites with Neural style, here are some examples: