Automating creation of new VM and pushing code to server

Automating creating of new VM and pushing code to server

Hi all,

I am writing a integration test for a client-server application. To simulate
the production server setup, and to update the code, I set up VMs on my
computer. The application runs on a Windows server, so I have to use systems
compatible with Windows.

STEP 1: Set up VM properly:

• Download the OVA file from MS

• create a new VM from the OVA file

• reset the mac id

• set a static dns ip address and a VM ip address

• open up the port 22

• install bitvise ssh server

• transfer public keys to the VM

• install python 3.5.2

• update pip

• install virtualenv systemwide

STEP 2: Deploy code to VM:

• create a requirements file using pip using: pip freeze -r requirements

• zip the entire repo using powershell using:

powershell.exe -nologo -noprofile -command "& { Add-Type -A 'System.IO.Compression.FileSystem'; [IO.Compression.ZipFile]::ExtractToDirectory('atu.zip', 'atu'); }"

• using pscp I transfer the files to the VM using:

pscp ./atu.zip ipython_user@10.15.33.31:/C:/Users/ipython_user/Desktop

• unzip the files again using powershell using:

powershell.exe -nologo -noprofile -command "& { Add-Type -A 'System.IO.Compression.FileSystem'; [IO.Compression.ZipFile]::ExtractToDirectory('atu.zip', 'atu'); }"

• delete the virtualenv folder called env in the folder using plink

• create a fresh env folder using using virtualenv: virtualenv env

• using the requirements.txt document, I install the requirements again by

direcatly using the pip installed in the virtualenv above using:
/path/to/env/Scripts/pip.exe install -r /path/to/requirements.txt

• run the file with the python distribution installed in the virtualenv

using:
/path/to/python file_name.py

Because I was trying to execute relatively complex commands, which in turn had
multiple quotes (double and single), spaces etc, I had to spend quite a bit of
time just debbuging how to run these commands over the n

Instead of using custom scripts which it seems that you are using, I would consider using open-source tools, similar to what Tensibai has suggested in his comments.

I would further split your two Tasks a bit more differently, such that Step 1 would be to just "Set up the VM", and Step 2 would be to "Provision the VM with required programs and configuration".

My 2-cents would be to use tools such as Packer and Ansible.

You would use (for example), the VirtualBox Packer builder, to stand up your VM, which would handle points 1-4 of your Step 1, and then use the Ansible-local provisioner, or something to this effect, to run a playbook, which would handle the rest of Step 1, and all of Step 2.

The above suggestion would make your life a lot easier, and would be able to abstract different OS's as well as you'd just write a different Packer template and Ansible task. With installing different Python versions / Tools, you'll just need to update the Ansible role / have several different roles with the specific tools.

There are of course, other tools such as Vagrant, Puppet, and Chef, which can handle what you'd like to achieve, so it's up to you find your preferred tool.

Mini breakdown of Ansible modules you can use:

• Open up port: win_firewal_rule_module

• Installing bitvise and python: win_chocolatey

• Setting up pip and virtual envs: pip

• Transferring public keys / requirements.txt etc: win_copy

Etc. The full list of Ansible Windows Modules can be found here.

StackExchange DevOps Q#3671, answer score: 2