Build a Super Resolution Image Server with Vultr's Cloud GPUs

Introduction

Real-Enhanced Super-Resolution Generative Adversarial Networks (Real-ESRGAN) is an open-source algorithm for image and video restoration. You can use Real-ESRGAN to restore or enhance low-resolution images by reducing the noise and upscaling them. This guide explains how to build a Super Resolution Image Web Application on a Vultr Cloud GPU using Streamlit and Real-ESRGAN.

Prerequisites

• Deploy a Vultr Cloud GPU running on Ubuntu 22.10 with at least 4 GB of GPU RAM. Upgrade your server's size if you want to enhance low-resolution images with a larger size or dimension.
• Update the Ubuntu Server.
• Create a non-root user with sudo access and log in with SSH.

Build a Super Resolution Image Web Application

1. Clone the Real-Enhanced Super-Resolution Generative Adversarial Networks (Real-ESRGAN) Repository. It contains the Machine Learning models to be used for the web application.

    $ git clone https://github.com/xinntao/Real-ESRGAN.git
   

2. Change the directory to the cloned repository Real-ESRGAN.

    $ cd Real-ESRGAN
   

3. Install Python3 Virtual Environment.

    $ sudo apt-get install -y python3.10-venv
   

4. Create a Virtual Environment for Real-ESRGAN.

    $ python3 -m venv esrgan-env
   

5. Activate the Virtual Environment.

    $ source ~/Real-ESRGAN/esrgan-env/bin/activate
   

6. Install BasicSR using pip. BasicSR is used for both training and inference.

    (esrgan-env)$ pip install basicsr
   

7. Install FaceXLib and GFPGAN. FaceXLib and GFPGAN are both used for the face enhancement feature.

    (esrgan-env)$ pip install facexlib
    (esrgan-env)$ pip install gfpgan
   

8. Install the required dependencies for Real-ESRGAN.

    (esrgan-env)$ pip install -r requirements.txt
   

9. Set up Real-ESRGAN by running the setup.py file.

    (esrgan-env)$ python3 setup.py develop
   

10. Install Streamlit. You will use the framework Streamlit for the Web Application's User Interface.

    (esrgan-env)$ pip install streamlit
    

11. Create a new folder named uploads. This folder is where the uploaded images will be saved.

    (esrgan-env)$ mkdir ~/Real-ESRGAN/uploads
    

12. Create a new folder named outputs. This folder is where the enhanced images will be saved.

    (esrgan-env)$ mkdir ~/Real-ESRGAN/outputs
    

13. Create a web-ui.py file.

    (esrgan-env)$ nano ~/Real-ESRGAN/web-ui.py
    

14. Populate the web-ui.py file with the codes below:

    Import the required modules:

    import streamlit as st
    import os
    import subprocess
    from PIL import Image

    Define the load_image() function. This function will open and identify the uploaded image file using Pillow, a fork of Python Imaging Library (PIL).

    @st.cache # initialize streamlit cache
    def load_image(image_file):
        img = Image.open(image_file)
        return img

    Define the main() function.

    def main():

    Set the Web Application's title and sub-header.

        st.title("Super Resolution Image Server")
        st.subheader("Upload Image to Enhance")

    Create a File Uploader widget (dropbox) that accepts image files with png, jpeg, and jpg extensions.

        image_file = st.file_uploader("Upload an Image:", type=['png', 'jpeg', 'jpg'])

    Check if the dropbox widget already contains a file, and then display the image's filename, MIME type, and the image itself with a 500px width.

        if image_file is not None:
            img_details = {"Image File Name": image_file.name, "Image File Type": image_file.type}
            st.write(img_details)
            img = load_image(image_file)
            st.image(img, width=500)

    Save the uploaded image file to the uploads folder you created in step 11.

            with open(os.path.join("uploads", image_file.name, ), "wb") as file:
                file.write(image_file.getbuffer())

    Real-ESRGAN has five image enhancement and restoration models: RealESRGAN_x4plus, RealESRNet_x4plus, RealESRGAN_x4plus_anime_6B, RealESRGAN_x2plus, and realesr-general-x4v3. Please see the Model Zoo Documentation for more information.

    Create a drop-down menu to select what model to use.

            model = st.selectbox(
                "Select Model to Use:",
                ("RealESRGAN_x4plus", "RealESRNet_x4plus", "RealESRGAN_x4plus_anime_6B", "RealESRGAN_x2plus", "realesr-general-x4v3")
            )
            st.write("You selected: ", model)

    Create a checkbox that will either enable or disable the face enhancement feature.

            face_enhance = st.checkbox("Enable Face Enhancement Feature")

    Create a button that will run the inference_realesrgan.py with the appropriate arguments when clicked. The arguments will be based on the chosen model and if the face enhancement feature is enabled or disabled. The enhanced image output will be placed in the outputs folder you created in step 12.

            if st.button("Enhance Image"):
                if model == "RealESRGAN_x4plus":
    
                    if face_enhance:
                        subprocess.call("python3 inference_realesrgan.py -n RealESRGAN_x4plus -i uploads/" + image_file.name + " -o outputs --face_enhance", shell=True)
                    else:
                        subprocess.call("python3 inference_realesrgan.py -n RealESRGAN_x4plus -i uploads/" + image_file.name + " -o outputs", shell=True)
                elif model == "RealESRNet_x4plus":
                    if face_enhance:
                        subprocess.call("python3 inference_realesrgan.py -n RealESRNet_x4plus -i uploads/" + image_file.name + " -o outputs --face_enhance", shell=True)
                    else:
                        subprocess.call("python3 inference_realesrgan.py -n RealESRNet_x4plus -i uploads/" + image_file.name + " -o outputs", shell=True)
                elif model == "RealESRGAN_x4plus_anime_6B":
                    if face_enhance:
                        subprocess.call("python3 inference_realesrgan.py -n RealESRGAN_x4plus_anime_6B -i uploads/" + image_file.name + " -o outputs --face_enhance", shell=True)
                    else:
                        subprocess.call("python3 inference_realesrgan.py -n RealESRGAN_x4plus_anime_6B -i uploads/" + image_file.name + " -o outputs", shell=True)
                elif model == "RealESRGAN_x2plus":
                    if face_enhance:
                        subprocess.call("python3 inference_realesrgan.py -n RealESRGAN_x2plus -i uploads/" + image_file.name + " -o outputs --face_enhance", shell=True)
                    else:
                        subprocess.call("python3 inference_realesrgan.py -n RealESRGAN_x2plus -i uploads/" + image_file.name + " -o outputs", shell=True)
                elif model == "realesr-general-x4v3":
                    if face_enhance:
                        subprocess.call("python3 inference_realesrgan.py -n realesr-general-x4v3 -i uploads/" + image_file.name + " -o outputs --face_enhance", shell=True)
                    else:
                        subprocess.call("python3 inference_realesrgan.py -n realesr-general-x4v3 -i uploads/" + image_file.name + " -o outputs", shell=True)

    Get the image's filename without the file extension.

                img_name = os.path.splitext(image_file.name)[0]

    Get the image's file extension.

                img_ext = os.path.splitext(image_file.name)[1]

    Open and identify the enhanced image file using Pillow.

                enhanced_image = Image.open('outputs/' + img_name + '_out' + img_ext)

    Display the enhanced image.

                st.image(enhanced_image, width=500)

    Create a Download button that will download the enhanced image when clicked.

                with open("outputs/" + img_name + "_out" + img_ext, "rb") as dl:
                    dl_btn = st.download_button(
                        label= "Download Enhanced Image",
                        data=dl,
                        file_name="enhanced_" + image_file.name,
                        mime=image_file.type,
                    )

    Run the main() function when executed.

    if __name__ == '__main__':
        main()
    

15. Save and exit the web-ui.py file by pressing Ctrl + X then Y then Enter.

16. Your web-ui.py file should look like this:

    import streamlit as st
    import os
    import subprocess
    from PIL import Image
    
    @st.cache
    def load_image(image_file):
        img = Image.open(image_file)
        return img
    
    def main():
    
        st.title("Super Resolution Image Server")
        st.subheader("Upload Image to Enhance")
    
        image_file = st.file_uploader("Upload an Image:", type=['png', 'jpeg', 'jpg'])
        if image_file is not None:
            img_details = {"Image File Name": image_file.name, "Image File Type": image_file.type}
            st.write(img_details)
            img = load_image(image_file)
            st.image(img, width=500)
    
            with open(os.path.join("uploads", image_file.name, ), "wb") as file:
                file.write(image_file.getbuffer())
    
            model = st.selectbox(
                "Select Model to Use:",
                ("RealESRGAN_x4plus", "RealESRNet_x4plus", "RealESRGAN_x4plus_anime_6B", "RealESRGAN_x2plus", "realesr-general-x4v3")
            )
            st.write("You selected: ", model)
            face_enhance = st.checkbox("Enable Face Enhancement Feature")
            if st.button("Enhance Image"):
                if model == "RealESRGAN_x4plus":
                    if face_enhance:
                        subprocess.call("python3 inference_realesrgan.py -n RealESRGAN_x4plus -i uploads/" + image_file.name + " -o outputs --face_enhance", shell=True)
                    else:
                        subprocess.call("python3 inference_realesrgan.py -n RealESRGAN_x4plus -i uploads/" + image_file.name + " -o outputs", shell=True)
                elif model == "RealESRNet_x4plus":
                    if face_enhance:
                        subprocess.call("python3 inference_realesrgan.py -n RealESRNet_x4plus -i uploads/" + image_file.name + " -o outputs --face_enhance", shell=True)
                    else:
                        subprocess.call("python3 inference_realesrgan.py -n RealESRNet_x4plus -i uploads/" + image_file.name + " -o outputs", shell=True)
                elif model == "RealESRGAN_x4plus_anime_6B":
                    if face_enhance:
                        subprocess.call("python3 inference_realesrgan.py -n RealESRGAN_x4plus_anime_6B -i uploads/" + image_file.name + " -o outputs --face_enhance", shell=True)
                    else:
                        subprocess.call("python3 inference_realesrgan.py -n RealESRGAN_x4plus_anime_6B -i uploads/" + image_file.name + " -o outputs", shell=True)
                elif model == "RealESRGAN_x2plus":
                    if face_enhance:
                        subprocess.call("python3 inference_realesrgan.py -n RealESRGAN_x2plus -i uploads/" + image_file.name + " -o outputs --face_enhance", shell=True)
                    else:
                        subprocess.call("python3 inference_realesrgan.py -n RealESRGAN_x2plus -i uploads/" + image_file.name + " -o outputs", shell=True)
                elif model == "realesr-general-x4v3":
                    if face_enhance:
                        subprocess.call("python3 inference_realesrgan.py -n realesr-general-x4v3 -i uploads/" + image_file.name + " -o outputs --face_enhance", shell=True)
                    else:
                        subprocess.call("python3 inference_realesrgan.py -n realesr-general-x4v3 -i uploads/" + image_file.name + " -o outputs",shell=True)
    
                img_name = os.path.splitext(image_file.name)[0]
                img_ext = os.path.splitext(image_file.name)[1]
    
                enhanced_image = Image.open('outputs/' + img_name + '_out' + img_ext)
                st.image(enhanced_image, width=500)
    
                with open("outputs/" + img_name + "_out" + img_ext, "rb") as dl:
                    dl_btn = st.download_button(
                        label= "Download Enhanced Image",
                        data=dl,
                        file_name="enhanced_" + image_file.name,
                        mime=image_file.type,
                    )
    
    if __name__ == '__main__':
        main()

Run the Super Resolution Image Web Application with Tmux

When you run your Streamlit-based web application on a normal SSH session, the Streamlit process closes when you exit the SSH session. To continuously run your web application even when you leave the SSH session, use tmux, a terminal multiplexer.

1. To create a Tmux process, run:

    (esrgan-env)$ tmux new -s esrgan-webapp

   You may change esrgan-webapp with any session name you prefer. Please see How to Install and Use Tmux for more information about Tmux.

2. Streamlit runs on port 8501 by default, so you need to allow port 8501 on the firewall.

    (esrgan-env)$ sudo ufw allow 8501
   

3. Launch the Super Resolution Image Web Application.

    (esrgan-env)$ streamlit run ~/Real-ESRGAN/web-ui.py

4. When you run Streamlit for the first time, you will be prompted to enter your email address. If you want to receive Streamlit updates, kindly enter your email address, or you can press Enter to skip it.

5. To detach from the Streamlit session, press Ctrl + B then D.

6. To view your Super Resolution Image Web Application, navigate port 8501 in your browser at your server's IP address. For example, http://192.0.1.2:8501.

Test the Super Resolution Image Web Application

1. In your browser, navigate port 8501 at your server's IP address. For example, http://192.0.1.2:8501.
2. In the file dropbox, upload any low-resolution image. The file dropbox only accepts PNG, JPG, and JPEG images. After uploading, the image will be displayed with a width of 500 pixels, along with its filename and MIME type.
3. Select the ML Model to be used for enhancing the image. Please see the Model Zoo Documentation for more information.
4. Select the Enable Face Enhancement Feature checkbox to enable face enhancement, or leave it unchecked otherwise.
5. Click the Enhance Image button to enhance the uploaded image's resolution. The application downloads the model's weights when you run it for the first time.
6. After enhancing the uploaded image, it will display the output image with a 500px width.
7. Click the Download Enhanced Image button to download the output image.
8. See the images below for an example.

Sample Execution:

Model: RealESRGAN_x4plus Face Enhancement: Disabled

Low-Resolution Image Input:

Enhanced Image:

You have successfully created a Super Resolution Image Server on a Vultr Cloud GPU using Streamlit and Real-ESRGAN.

More Information

To learn more about Streamlit and Real-ESRGAN, please see these resources:

• Streamlit Documentation
• Real-ESRGAN GitHub Repository
• Real-ESRGAN Models Documentation