Untamed Creativity: Become an AI Artist at Zero Costs
Craft Your Own Masterpieces Using Stable Diffusion
Artificial intelligence (AI) is becoming increasingly prevalent in our lives, from helping us find the best route home to suggesting what we should buy next.
But what about AI in the world of art? With Stable Diffusion, you can become an AI artist and create your own masterpieces.
With the advent of Stable Diffusion, an open source Generative AI model (GAN), becoming an AI artist has become an exciting and accessible endeavor. We will explore in the following the potential of stable diffusion in enabling individuals to express their creativity and transform the art world.
And the best:
It is installed locally on your computer and it comes without any subscription fee!
What is Stable Diffusion?
Stable Diffusion is a deep learning, text-to-image model released in 2022, primarily used to generate detailed images conditioned on text descriptions. It was developed by Stability AI in collaboration with academic researchers and non-profit organizations. The model is a latent diffusion model, a type of deep generative neural network.
The model can generate new images from scratch using a text prompt describing elements to be included or omitted from the output. It can also be used for guided image synthesis, inpainting, outpainting, and generating image-to-image translations guided by a text prompt. Stable Diffusion is an open-source machine learning model that can be run on consumer hardware equipped with a modest GPU with at least 4 GB VRAM.
There are two critical differences that set Stable Diffusion apart from other popular AI art generators:
- Stable Diffusion is open-source and can be run on your own PC (or Mac).
- The model supports a command-line interface (CLI), which can be less intuitive than graphical interfaces but allows for more customization.
Stable Diffusion has an extensive dataset, the 2b English language label subset of LAION 5b, which is a general crawl of the internet created by the German charity LAION.
To use Stable Diffusion, you can install it and run it through a command-line interface. There are also web-based applications and graphical user interfaces (GUIs) available for easier use.
Using Stable Diffusion means, you can experiment with various settings, such as:
- Painter’s Palette Button: Applies a random artistic style to your prompt .
- Sampling Steps: The number of times the image will be refined before you receive an output. More steps generally yield better results, but there are diminishing returns.
- Sampling Method: The underlying math governing how sampling is handled. Euler_a and PLMS are popular options.
- Restore Faces: Uses GFPGAN to fix uncanny or distorted faces.
- Batch Size: The number of “batches”. It is recommended to keep this at 1 unless you have an enormous amount of VRAM (> 8 GB).
- CFG Scale: How closely Stable Diffusion will follow the prompt you give it. Larger numbers mean it follows it very carefully, whereas lower numbers give it more creative freedom.
The copyright for using Stable Diffusion-generated images varies from jurisdiction to jurisdiction.
How to use Stable Diffusion
Using Stable Diffusion is relatively easy.
First, you need to install the software on your computer.
Once installed, you can use the interface via a web browser to select the input image and set the parameters for the diffusion process.
After that, you can simply click “generate” and let Stable Diffusion do the rest.
One of the great things about Stable Diffusion is that it can be used locally, meaning you don’t need to rely on cloud-based services that can be of limited availability and expensive.
By using Stable Diffusion locally, you can take advantage of the power of your own computer and create stunning images quickly and efficiently without any additional costs.
Installation Preparation
The installation is relatively easy, due to the extensive suppport by the Open Source community. For the most components, there are packages and installers available which makes it convenient for you to install Stable Diffusion in your own environment:
What you need
A medium sized PC or Notebook is generally sufficient. The most work is being done by the GPU which should have at least 4GB of VRAM.
Of course, the better your hardware, the faster your results become.
From a software side, Stable Diffusion can be downloaded from GitHub here: Stable Diffusion WebUI on GitHub
Moreover, Stable Diffusion uses PyTorch, which needs to be installed. However, the repository checks for the installation of PyTorch and downloads and installs it automatically in case it’s not there yet.
Since this tutorial is focusing on a Windows 11 setup with a NVIDIA GPU, we’ll list the corresponding components only (however there are plenty tutorials and available for a Linux/Radeon setup on the internet):
- Python for Windows
- Git version control system for Windows
- Microsoft Visual C++ Redistributable for Visual Studio 2022
- NVIDIA Compute Unified Device Architecture (CUDA)
Installation Steps
First install Python in a 3.10.x version from the official webpage. For the sake of simplicity, please stick with the 3.10.x version, since I’ve personally encountered several installation/running issues by using a later version.
Second, install the latest version of Git from the official website
As a third step, install the MS Visual C++ Runtime (if not already installed since the runtime is required by a lot of other applications)
Finally, install NVIDIA CUDA Toolkit to enable GPU support for Stable Diffusion
After the installation is completed, check if CUDA is installed correctly by retrieving its version:
nvidia-smi.exeThe output of this command gives you some information about the GPU and your CUDA version in the upper right corner.
It is important to install CUDA before you start installing Stable-Diffusion in the next steps, since it selects the appropriate PyTorch version based on your installed CUDA version.
Now, you’re ready to download the Stable Diffusion repository from GitHub via the following command:
git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.gitMove to the stable-diffusion-webui folder in the downloaded repository and execute the following script:
.\webui.bat
This script will download all missing packages and components (including PyTorch). Depending on your internet connection, this may take a while (> 5 GB of data to be downloaded for the first time).
Congratulations, the installation is finished. Now start Stable Diffusion and open your web browser and navigate to the following url:
.\webui-user.batDepending on your GPU (more specifically the amount of VRAM) you can tweak the overall performance via command line arguments. In any ways it is recommended to use the argument
-- xformersto lower the RAM consumption while accelerating the processing for NVIDIA graphics cards.
Moreover, you can use additionally use
--medvram
--lowvramin case your GPU has less than 10 GB of VRAM (medvram) or less than 6 GB (lowvram).
Once your Stable Diffusion instance is up and running, you can use a web browser and navigate to the following URL where you should see the Web Interface then:
http://127.0.0.1:7860
The Results
You have plenty possibilities to create pictures through multimodality.
As a start I recommend to use the text-to-image (via the “txt2img” tab) where you can describe the picture you want to generate.
In our example we simply ask for a photo of a burger.
And as you can see, the result is by default quite impressive.
Conclusion and further references
As stated in the beginning - if you’re looking to become an AI artist, look no further than Stable Diffusion.
With its powerful deep learning algorithm and easy-to-use interface, you can create stunning, high-quality images with ease. Start experimenting with Stable Diffusion and see what kind of masterpieces you can create.
But, please don’t get discouraged by the vast variety of parameters. Especially in the beginning you might get overwhelmed by the options the model provides and hence your results might become disappointing.
To help you get started in tweaking and improving your pictures here’s a great resource to do so:
And now, enjoy your adventure to become an AI artist!
About Tobias Faiss
Tobias is a Senior Engineering Manager, focusing on applied Leadership, Analytics and Cyber Resilience. He has a track record of 18+ year in managing software-projects, -services and -teams in the United States, EMEA and Asia-Pacific. He currently leads several multinational teams in Germany, India, Singapore and Vietnam. Also, he is the founder of the delta2 edventures project where its mission is to educate students, IT professionals and executives to build a digital connected, secure and reliable world and provides training for individuals.
Tobias’ latest book is ‘The Art of IT-Management: How to Successfully Lead Your Company Into the Digital Future’. You can also contact him on his personal website tobiasfaiss.com
