A new tool called Otto came out recently. It’s a tool from Hashicorp. At the project homepage we can see the headline “meet the successor to Vagrant”. I know most of you are familiarized with Vagrant, but for those that are not: Vagrant is a tool to manage virtual development environments using VirtualBox or VMWare, for example.
With Vagrant you can basically create a virtual machine to develop your application and make sure that everyone on your team shares the same configurations and tools to work on your application. Hashicorp built Vagrant and Otto, but these are not the only tools they built. They actually have a set of tools to help us develop and deploy software better. These tools together forms what they call the “Hashicorp ecosystem”.
Hashicorp ecosystem
Before we start talking about Otto, let’s understand this ecosystem a bit more. I’m just going to describe some of the tools here. To get a more in-depth look, check the documentation of each tool.
Vagrant
As I said, Vagrant is a tool for building development environments that can be shared across your team as portable VMs.
Packer
Packer is a tool for generating “build artifacts”, such as Amazon’s AMI, OVF for VirtualBox, Docker containers etc. The idea is that you can generate an “image” of your application installed and configured that can be easily deployable to a cloud platform.
Consul
Consul is a tool for service discovery, service configuration and a few other things. Basically, you can use Consul to make sure your deployable artifacts are going to be ok when they are deployed, and you can also store some configurations on your Consul server to share across your applications, which can make it easier to change, but you can do more with Consul. It is also responsible for resolving your applications dependencies hosts (internal DNS), let’s say you have a Redis dependency, then you request for redis.service.consul and the Consul server will respond with the right IP address.
Terraform
Terraform it’s a tool for building infrastructure in a smart way. Since Terraform can build, destroy and modify servers and configurations, you use it to deploy the Packer generated images to new cloud servers/clusters. It manages your cloud infrastructure for you.
Here is the “Hashicorp ecosystem” workflow in a picture:
What does it have to do with Otto?
You’re probably asking yourself why I talked about this workflow, right? “Is he trying to sell these tools?”. Actually, these tools I mentioned are all open source. Hashicorp’s only commercial product is Atlas, it integrates all the other products so you can turn your application delivery in a collaborative process.
I talked about all these tools because Otto is an abstraction of these tools. Otto knows about your development environment, so it can easily build a vagrant box based on your application’s dependencies. It knows how to deploy your application because it uses Consul, Packer and Terraform to build a versioned and realiable infrastructure and build process.
The Successor of Vagrant
They wrote a dedicated section in the documentation to explain why and how Otto is going to replace Vagrant, so I’m not going to focus on it here. However, I want to highlight some points that I like.
As they say, the Vagrantfile describes your Vagrant box (VM), how to setup all the services you want and so on. On the other hand, the Appfile in Otto describes your application and its direct dependencies. From that, Otto can build your Vagrant box. That’s awesome! It means our application and its dependencies are the focus now, not the Vagrant box. This is also how Otto knows how to deploy the application, a thing Vagrant can’t do for us because it only knows how to setup a single machine.
Built for Microservices
Nowadays, applications are way more complex than a few years ago. We no longer have a single web server that interacts with the database and that’s it. Our applications have many dependencies, like an Elasticsearch server, a
Redis server, a worker server.. or many small applications like an Auth API that centralizes all authentication of your applications. Building a development environment with all these dependencies is a pain. And if we try to use many vagrant VMs locally we quickly run out of memory.
Otto knows our application’s dependencies and the dependencies of our dependencies, so it knows how to build a VM for our application with every service we need on it. It also knows that when deploying (depending on the flavor we choose for our application) that it needs to split these services in different servers and so on. I know, that sounds like magic, right?
Right now, we have two flavors of infrastructure: simple and vpc-public-private. I’m going to give a brief explanation here, but check the docs for a more broad view.
simple flavor
It builds a Consul server for application discovery that basically will serve as a DNS service for our application. It also creates a Packer server that will build the AMI that we can deploy to a new server. The Packer server will be destroyed at the end of the build process, so you don’t need to worry about it being there just increasing your bill. Worth saying that this option was built for demonstration purposes. It will do in the cloud what it does on your development environment: build your application and its dependencies in a single server.
vpc-public-private flavor
The VPC is a more robust option. That being said, it’s more expensive and takes longer to build. It creates a VPC with public and private subnets and it uses 3 Consul servers (a cluster) for high availability. It also distributes your application dependencies in dedicated servers that can be handled separately.
Ok, ok.. it’s time to build something!
First of all, Otto has a great getting started guide. You should follow it for a better explained tutorial. What I’m going to do here is basically start a fresh application that you can enter your name and submit it.
Let’s create a new Laravel application:
laravel new otto-laravelAfter that, we need to configure the Otto Appfile the will describe the application.
The Appfile
application {
name = "otto-laravel"
type = "php"
}
project {
name = "otto-laravel"
infrastructure = "otto-laravel"
}
infrastructure "otto-laravel" {
type = "aws"
flavor = "simple"
}We can now compile our application, cd into the root of the app and then run the otto compile.
Now Otto knows about our application. So we can build the development environment using vagrant.
Development Environment
To build your development VM, just run:
otto devThis command will build your Vagrant box (will ask to install it if you don’t have it already). It can take a while to provision your VM. After it finishes, we can SSH into the VM and start our server:
otto dev ssh
php artisan server --host=0.0.0.0 --port=5000
> Laravel development server started on http://0.0.0.0:5000/We can access it using the IP of our VM like so (you can get the VM IP address using otto dev address)
Laravel welcome page on Otto
Laravel welcome page on Otto
Let’s build the app, first add the routes:
Input::get('name')]);
});Then, let’s add the views:
resources/views/welcome.blade.php
@extends('_layout')
@section('content')
Write your name here
@stopresouces/views/user.blade.php
@extends('_layout')
@section('content')
Hello {{ '{{ $name ' }}}}
@stop
otto dev ssh php artisan serve --host=0.0.0.0 --port=5000
> Laravel development server started on http://0.0.0.0:5000/Laravel application running locally
Laravel application running locally
That’s it for the development environment. Now, you have your vagrant box ready to develop your application. You can run any vagrant command using otto, like so: otto dev vagrant ANY_VAGRANT_COMMAND.
Infrastructure
This process will require you to install Terraform, Otto is going to do that for you. Be aware that you might be charged by AWS for a few dollars by running this example. Don’t worry, we are going to destroy everything at the end so it won’t keep charging you. This example fits in the free tier of AWS.
Since we have finished our really small application, it’s time to setup our infrastructure. With Otto it’s a really simple thing to do. Just run otto infra, paste your cloud credentials (I’m using AWS for this demo) and you’re done. Otto knows what to build and how.
Otto will setup a Consul server for us that will be used for service discovery. We are going to explain a bit later.
Build Artifact
We can now generate a build artifact (it will use Packer behind the scenes) that Otto can deploy using Terraform. To do that, just run otto build. It will create a Packer server on AWS and install our application for us (it will run composer), then it will generate an AMI (Amazon’s image with our application installed) that we are going to deploy using Terraform in the next step.
Deploy!
After the AMI is generated, we can now deploy our application. Run otto deploy. At the end of this process, Otto will print the URL to our application deployed on AWS.
Application Deployed on AWS
Application Deployed on AWS
That’s awesome! Unfortunately, there is no way to tell our infrastructure what the webroot of our application is (at least at the time I’m writing this), Otto is still in development and it’s evolving really quickly, so expect it in the next few days or weeks.
Dependencies
For the first version of this blog post I was going to build a small app that keeps a count per access on Redis, but I couldn’t get the Redis dependency to work on AWS. I found an issue in the official repo, so at least I’m not alone here. If you want to try to build some dependencies, here is how you can configure your Appfile:
application {
name = "otto-laravel"
type = "php"
dependency {
source = "./otto/redis"
}
}The source of your dependencies can be local or a github URL, like: "github.com/tonysm/otto-laravel/otto/redis". Whenever you change your Appfile, you have to recompile it before seeing any changes. But first, we need to compile the Redis app. Create these folders otto/redis inside your application root path, then create a new Appfile in there with:
application {
name = "redis"
type = "docker-external"
}
customization "docker" {
image = "redis:3.0"
run_args = "-p 6379:6379"
}Now, compile the Redis app running otto compile inside the otto/redis folder. It will generate the .ottoid file and update the .otto structure with the new dependency. You need the .ottoid so your main application can identify the dependencies. After that, get back to the root path and run otto compile again. If you have the vagrant VM running, make sure you destroy it and recreate it again with otto dev destroy and otto dev. Now, you can access your Redis service inside your VM by hostname redis.service.consul, as you can see:
otto dev ssh
dig +noall +answer redis.service.consul
> redis.service.consul. 0 IN A 10.0.2.15The Consul foundation installs a local Consul server in your development environment and configures it to respond to *.consul DNS queries.
Let’s use it. First, edit the welcome view:
@extends('_layout')
@section('content')
Counts: {{ '{{ $' }}counts }}
@stopNow, let’s change the home route to keep a count of accesses:
Redis::incr('counts')]);
});We need to configure the Redis host in the .env file, add this line to it:
REDIS_HOST=redis.service.consulIn the config/database.php file we can use it:
[
'cluster' => false,
'default' => [
'host' => env('REDIS_HOST', 'locahost'),
'port' => 6379,
'database' => 0,
],
],
];Start the PHP server using Artisan inside the VM:
otto dev ssh
php artisan serve --host=0.0.0.0 --port=5000
> Laravel development server started on http://0.0.0.0:5000/If everything went well, you will see the counts increasing everytime you access the route:
Laravel Redis application locally
Laravel Redis application locally
I tried building and deploying this application, but somehow the Redis docker service wasn’t being installed on the EC2 server (remember the simple flavor infrastructure).
Conclusion
Now, we can teardown our deploy, infrastructure and development environments, doing so:
otto deploy destroy
otto infra destroy
otto dev destroyMake sure you run these command in this order. And done! Otto cleaned up everything it created on AWS and locally (Vagrant).
I struggled with it a bit. The build process failed a few times saying it couldn’t install some php extension, but worked in the next builds. Also, it isn’t possible (AFAIK) to setup the root path of the apache virtualhost, which is awful. But I’m sure it’s going to get fixed soon.
Otto is not ready for production yet, but I have to say: well done, Hashicorp! I have to admit that I didn’t know any of these tools before (except for Vagrant) and I’m very impressed! Otto looks great and I can’t wait to use it in a real project. I’ll update this blogpost as soon as I get the Redis example up and running in the cloud.
Member discussion