🎨 texture-synthesis
A light Rust API for Multiresolution Stochastic Texture Synthesis [1], a non-parametric example-based algorithm for image generation.
The repo also includes multiple code examples to get you started (along with test images), and you can find a compiled binary with a command line interface under the release tab.
Also see our talk More Like This, Please! Texture Synthesis and Remixing from a Single Example which explains this technique and the background more in-depth:
Features and examples
1. Single example generation
Generate similar-looking images from a single example.
API - 01_single_example_synthesis
use texture_synthesis as ts;
CLI
cargo run --release -- --out out/01.jpg generate imgs/1.jpg
You should get the following result with the images provided in this repo:
2. Multi example generation
We can also provide multiple example images and the algorithm will "remix" them into a new image.
API - 02_multi_example_synthesis
use texture_synthesis as ts;
CLI
cargo run --release -- --rand-init 10 --seed 211 --in-size 300x300 -o out/02.png --debug-out-dir out generate imgs/multiexample/1.jpg imgs/multiexample/2.jpg imgs/multiexample/3.jpg imgs/multiexample/4.jpg
You should get the following result with the images provided in this repo:
3. Guided Synthesis
We can also guide the generation by providing a transformation "FROM"-"TO" in a form of guide maps
API - 03_guided_synthesis
use texture_synthesis as ts;
CLI
cargo run --release -- -o out/03.png generate --target-guide imgs/masks/2_target.jpg --guides imgs/masks/2_example.jpg -- imgs/2.jpg
NOTE: Note the use of --
to delimit the path to the example imgs/2.jpg
, if you don't specify --
, the path
to the example will be used as another guide path and there won't be any examples.
You should get the following result with the images provided in this repo:
4. Style Transfer
Texture synthesis API supports auto-generation of example guide maps, which produces a style transfer-like effect.
API - 04_style_transfer
use texture_synthesis as ts;
CLI
cargo run --release -- --alpha 0.8 -o out/04.png transfer-style --style imgs/multiexample/4.jpg --guide imgs/tom.jpg
You should get the following result with the images provided in this repo:
5. Inpaint
We can also fill-in missing information with inpaint. By changing the seed, we will get different version of the 'fillment'.
API - 05_inpaint
use texture_synthesis as ts;
CLI
Note that the --out-size
parameter determines the size for all inputs and outputs when using inpaint!
cargo run --release -- --out-size 400 --inpaint imgs/masks/3_inpaint.jpg -o out/05.png generate imgs/3.jpg
You should get the following result with the images provided in this repo:
6. Tiling texture
We can make the generated image tile (meaning it will not have seams if you put multiple images together side-by-side). By invoking inpaint mode together with tiling, we can make an existing image tile.
API - 06_tiling_texture
use texture_synthesis as ts;
CLI
cargo run --release -- --inpaint imgs/masks/1_tile.jpg --out-size 400 --tiling -o out/06.bmp generate imgs/1.jpg
You should get the following result with the images provided in this repo:
7. Combining texture synthesis 'verbs'
We can also combine multiple modes together. For example, multi-example guided synthesis:
Or chaining multiple stages of generation together:
For more use cases and examples, please refer to the presentation "More Like This, Please! Texture Synthesis and Remixing from a Single Example"
Command line binary
- Download the binary for your OS, or install it from source,
cargo install --path=.
. Note: if you want to show a progress window you will need to enable theprogress
feature, eg.cargo install --path=. --features="progress"
- Open a terminal
- Navigate to the directory where you downloaded the binary, if you didn't just
cargo install
it - Run
texture_synthesis --help
to get a list of all of the options and commands you can run - Refer to the examples section in this readme for examples of running the binary
Limitations
- Struggles with complex semantics beyond pixel color (unless you guide it)
- Not great with regular textures (seams can become obvious)
- Cannot infer new information from existing information (only operates on what’s already there)
- Designed for single exemplars or very small datasets (unlike Deep Learning based approaches)
Links/references
[1] [Opara & Stachowiak] "More Like This, Please! Texture Synthesis and Remixing from a Single Example"
[2] [Harrison] Image Texture Tools
[3] [Ashikhmin] Synthesizing Natural Textures
[4] [Efros & Leung] Texture Synthesis by Non-parametric Sampling
[5] [Wey & Levoy] Fast Texture Synthesis using Tree-structured Vector Quantization
[6] [De Bonet] Multiresolution Sampling Procedure for Analysis and Synthesis of Texture Images
[7] All the test images in this repo are from Unsplash
Contributing
We welcome community contributions to this project.
Please read our Contributor Guide for more information on how to get started.
License
Licensed under either of
- Apache License, Version 2.0, (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
- MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)
at your option.
Contribution
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.