How to Use Vultr Object Storage in Python

Published on March 28, 2022•Updated on November 21, 2023

Introduction

Vultr Object Storage is a highly scalable storage solution. It gives you the flexibility to add scalable storage on demand and manage it through S3 API. It provides a platform to store large quantities of files without worrying about the underlying complexities of handling a filesystem. It ensures the safety of your files by keeping them redundantly across multiple locations, which makes it a reliable storage solution.

You can use object storage in your Python applications using Boto3, a Python library that enables developers to interact with S3 API to manage object storage programmatically.

This article demonstrates how to perform common operations such as creating or deleting buckets and objects programmatically using Python and Boto3 SDK.

Prerequisites

Vultr Object Storage configured in your account.
The access key and secret key for your Object Store.
Python, version 3.6 or later.

Install Boto3 SDK

Install boto3 using Python package manager.

pip install boto3

Create Boto3 Client

To use Vultr Object Storage with Boto3, you need to create a Boto3 client instance with the S3 Credentials. In the following steps, you use this client to perform operations on your object storage. Replace the first three variables with your credentials.

import boto3

hostname = "HOSTNAME"
secret_key = "SECRET_KEY"
access_key = "ACCESS_KEY"

session = boto3.session.Session()
client = session.client('s3', **{
    "region_name": hostname.split('.')[0],
    "endpoint_url": "https://" + hostname,
    "aws_access_key_id": access_key,
    "aws_secret_access_key": secret_key
})

Bucket Operations

Buckets are like folders in a traditional filesystem. You need to have at least one bucket in your object storage to upload files on it. The name of your bucket should be unique region-wide, you can achieve that by adding a random prefix to your name, for example, 11293x-name.

List All Buckets

Use the list_buckets() method to retrieve a list of buckets in your object storage.

response = client.list_buckets()
print(response)

Create a New Bucket

Use the create_bucket() method to create a new bucket in your object storage.

response = client.create_bucket(Bucket='example-bucket')
print(response)

Delete an Empty Bucket

Use the delete_bucket() method to delete an empty bucket from your object storage.

response = client.delete_bucket(Bucket='example-bucket')
print(response)

Delete a Non-Empty Bucket

To delete a non-empty bucket, you need to delete all objects inside it by iterating over the list of objects available.

object_list = client.list_objects(Bucket='example-bucket')

for object in object_list['Contents']:
    client.delete_object(Bucket='example-bucket', Key=object['Key'])

response = client.delete_bucket(Bucket='example-bucket')
print(response)

Object Operations

Objects stored in buckets, like files in a traditional filesystem. In S3 API, the key is the name, and the body is the byte content of the file.

List All Objects

Use the list_objects() method to retrieve a list of objects in a bucket.

response = client.list_objects(Bucket='example-bucket')
print(response)

Download an Object

Use the download_file() method to download an object from a bucket.

response = client.download_file('example-bucket', 'file.ext', 'path/to/local-file/name.ext')
print(response)

Upload a New Object

Use the put_object() method to upload a new file to a bucket.

with open('path/to/file/name.ext', 'rb') as file:
    response = client.put_object(Bucket='example-bucket', Key='file.ext', Body=file, ACL='private')

print(response)

Set the ACL parameter to public-read if you want the object to be publicly accessible.

Delete an Object

Use the delete_object() method to delete an object from a bucket.

response = client.delete_object(Bucket='example-bucket', Key='path/to/object/name.ext')
print(response)

Create Pre-Signed URLs

A pre-signed URL grants temporary access of an object to a user without credentials or permission to access the object.

Create a Pre-signed URL to Download a File

Use the generate_presigned_url() method to generate a pre-signed URL to download a file.

response = client.generate_presigned_url(ClientMethod='get_object',
                                Params={'Bucket': 'example-bucket',
                                        'Key': 'file.ext'},
                                ExpiresIn=300)

print(response)

The ExpiresIn parameter accepts an integer value defining the number of seconds the pre-signed URL is valid for.

Create a Pre-signed URL to Upload a File

Use the generate_presigned_url() method to generate a pre-signed URL to upload a new file.

response = client.generate_presigned_url(ClientMethod='put_object',a
                                Params={'Bucket': 'example-bucket',
                                        'Key': 'new-file.ext'},
                                ExpiresIn=300)

print(response)