How to Use Vultr’s Miniconda Marketplace App
Introduction
Miniconda is a free minimal installer for conda. It is a small, bootstrap version of Anaconda that includes only conda, Python, the packages they depend on, and a small number of other useful packages.
This guide shows how to deploy and secure a Miniconda instance on Vultr. It also shows the basic usage of Miniconda and usage examples.
Prerequisites
- Working knowledge of Python.
Deploy the instance
Deploy your Miniconda instance using Vultr Marketplace App. You need a "Cloud GPU" instance to run this app. The deployment may take a few minutes to fully complete and you can follow the initialization process on the instance console.
After the deployment, log in on the server via SSH.
Update and reboot the machine to apply the updates
# apt-get update && apt-get dist-upgrade -y
# anaconda/bin/conda init
# anaconta/bin/conda update conda
# rebootSecuring the instance
If you don't have an SSH key pair installed yet, create if you don't have one and install it on your instance.
Configure SSH to not accept password logins. You need your SSH key pair installed because you will not be able to log in via SSH with your password anymore. Open the file /etc/ssh/sshd_config with your favorite editor (vim, for example), uncomment the following line
PasswordAuthentication noand restart the SSH server
# systemctl restart sshIt is a good policy to close all ports on the instance firewall and open only the required ports like SSH
# ufw reset
# ufw enable
# ufw default allow outgoing
# ufw default deny incoming
# ufw allow ssh/tcpThe last command makes the firewall accept connections on port 22 from any host. If you have a static IP, you can change the last line to only allow connections from that specific IP
# ufw allow proto tcp from YOUR_IP to INSTANCE_IP port 22ufw supports connection rate limiting, which is useful for protecting against brute-force login attacks. When a limit rule is used, ufw will normally allow the connection but will deny connections if an IP address attempts to initiate 6 or more connections within 30 seconds.
# ufw limit ssh/tcpYou can see the firewall rules using the command below
# ufw status verbose
Status: active
Logging: on (low)
Default: deny (incoming), allow (outgoing), disabled (routed)
New profiles: skip
To Action From
-- ------ ----
22/tcp LIMIT IN Anywhere
22/tcp (v6) LIMIT IN Anywhere (v6)Using Miniconda
Managing virtual environments
It is recommended to use virtual environments while working on Python projects. Create a virtual environment with Miniconda with the command
# conda create --name myvenvor create the virtual environment specifying the Python version
# conda create --name myvenv python=3.10You can also clone an existing environment
# conda create --name clonevenv --clone myvenvAfter creating a virtual environment, it needs to be activated
# conda activate myvenvIf you need to deactivate an environment
# conda deactivateTo remove a virtual environment
# conda env remove -n myvenvInstalling packages
Install a package with the command
# conda install pandasYou can view the installed packages with
# conda listTo remove an installed package
# conda remove pandasThe commands above are executed on the active virtual environment. You can also execute those commands on another virtual environment by using the parameter -n
# conda install -n anothervenv pandasKNN classification example
Create an environment for the example
# conda create --name knn_example
# conda activate knn_exampleInstall scikit-learn library
# conda install scikit-learnCreate the file knn_example.py with the following code
from sklearn import datasets
from sklearn.model_selection import train_test_split
from sklearn.neighbors import KNeighborsClassifier
from sklearn import metrics
# Load data
iris = datasets.load_iris()
print("Features:", iris.feature_names)
print("Labels:", iris.target_names)
print("Data size:", iris.data.shape)
print("Examples:\n", iris.data[0:3])
# Split train and test instances. 70% training, and 30% test
X_train, X_test, y_train, y_test = train_test_split(iris.data, iris.target, test_size=0.3)
# Model fit
knn = KNeighborsClassifier(n_neighbors=5).fit(X_train, y_train)
# Predict
y_pred = knn.predict(X_test)
# Result
print("Accuracy:", metrics.accuracy_score(y_test, y_pred))This code loads the iris flower dataset and prints some information about this dataset. The dataset is split randomly into train and test subsets. The train portion is used to train the KNN classifier and the test portion is used to validate the model by comparing the predicted result given by the model with the real labels.
Run the code with
# python knn_example.py
Features: ['sepal length (cm)', 'sepal width (cm)', 'petal length (cm)', 'petal width (cm)']
Labels: ['setosa' 'versicolor' 'virginica']
Data size: (150, 4)
Examples:
[[5.1 3.5 1.4 0.2]
[4.9 3. 1.4 0.2]
[4.7 3.2 1.3 0.2]]
Accuracy: 0.9777777777777777Using Jupyter Notebook
Create a virtual environment and install Jupyter
# conda create --name jupyter
# conda activate jupyter
# conda install jupyterBy default, Jupyter runs a webserver only accessible from localhost. To be able to use it from a remote machine, you can use SSH port forwarding.
Connect to the Miniconda instance using SSH and port forwarding, port 8888 in this case which is the port Jupyter runs by default
$ ssh -L 8888:127.0.0.1:8888 root@INSTANCE_IPOnce in the Miniconda instance, start Jupyter Notebook with the command
# jupyter nbclassic --allow-root --no-browser
...
To access the notebook, open this file in a browser:
file:///root/.local/share/jupyter/runtime/nbserver-17248-open.html
Or copy and paste one of these URLs:
http://localhost:8888/?token=864fa232b1c37127616370df9e6bf1f867658c240ac9d97f
or http://127.0.0.1:8888/?token=864fa232b1c37127616370df9e6bf1f867658c240ac9d97fCopy the URL and paste it into the browser on your local machine.
The security of this connection is granted by SSH encryption as we are forwarding connections on port 8888 through the SSH tunnel. Alternatively, you can make Jupyter available publically using HTTPS with password protection.
Generate the Jupyter configuration file
# jupyter server --generate-configDefine a secure password
# jupyter server passwordYou can run this command anytime you need to change the password.
Generate a self-signed SSL certificate with OpenSSL to secure the HTTPS connections
# openssl req -x509 -newkey rsa:4096 -keyout /etc/ssl/private/server.key -out /etc/ssl/server.crt -days 365 -nodesor if you have a domain, use a certificate authority like Let's Encrypt to issue the certificate.
Edit the following lines on the Jupyter configuration file /root/.jupyter/jupyter_server_config.py
c.ServerApp.allow_password_change = False
c.ServerApp.allow_root = True
c.ServerApp.certfile = u'/etc/ssl/server.crt'
c.ServerApp.keyfile = u'/etc/ssl/private/server.key'
c.ServerApp.ip = '0.0.0.0'
c.ServerApp.open_browser = False
c.ServerApp.port = 8888Open the Jupyter port on the firewall
# ufw allow 8888/tcpStarting the Jupyter server with no parameter is enough as the needed parameters are already set on the configuration file
# jupyter nbclassicAnyone on the internet can see your server running on https://INSTANCE_IP:8888/, but the access is protected with your password.
Notice that using self-signed SSL certificates will trigger a warning on your browser when you access your Jupyter webserver. Don't be afraid of those warnings, you can bypass them.
Jupyter Lab is also available by appending /lab in the URL
http://127.0.0.1:8888/lab?token=864fa232b1c37127616370df9e6bf1f867658c240ac9d97f
or
https://INSTANCE_IP:8888/lab
depending on which method you are using.
Conclusion
This guide covered how to deploy and secure a Miniconda instance, manage Python virtual environments, install, and uninstall packages using Miniconda. It also showed two usage examples, KNN Classification using scikit-learn and how to use Jupyter Notebooks.