Recreating AWS Fargate (lite)

I’ve recent been on a spree building out my infrastructure as docker containers. I’m often torn in the debate between the cloud and on-prem infrastructure. I think the reason for that is two fold. One: I really like the tangible nature of the on-prem hardware. Seeing the servers, storage, and network gear and thinking about all the bits flying all over the place gives me comfort for some reason. Two: I like understanding the logic of how a system works. In the on-prem world with traditional silo’ed infrastructure you have to understand the inner workings to create automation on top of it. I love knowing the inner workings of IT systems.

Lately however I’ve been using more and more cloud providers. I played around with AWS Fargate where they manage the cluster of container hosts for you. You can set auto scale rules so it adds more computer power as it creates more replicas of your container. It will even register and deregister the containers from the elastic load balancer, make health decisions and “self heal” your application. It integrates with Route 53 for your DNS entries. It redirects your log files and monitors several metrics for you. I can only imagine Amazon invested a lot in this system so it may be lofty of me to want to recreate this. I’d really like to remain provider agnostic but its hard when the ecosystem of the provider works so well together right out of the box.

Because my credit card can only take so much from Amazon experiments I decided to try and create something similar on Digital Ocean. I’m using Ansible to spin up swarm worker nodes, from the inception of the VM to updates and configuration and joining the swarm. That is tied together with PowerShell. There may be a better way but its the scripting language I’m most familiar with and it seems to work quite well on Linux. I’m using influxDB and telegraf to gather performance metrics, I hope to have a query drive scaling decisions. I still need to figure out the load balancing aspect and probably others that I haven’t thought of. Here’s the main part that will drive the creation of new droplets on DigitalOcean.

#Requirements: Digital Ocean command line, powershell running on linux, ansible
#I can't seem to find a way to find droplets associated with a project so I'm kind of cheating and finding droplets with swarm in the name
$droplets = doctl compute droplet list --output json | ConvertFrom-Json  
$swarmIDs = New-Object System.Collections.ArrayList
foreach($droplet in $droplets){
    if($droplet.name -like 'swarm*'){
    $swarmIDs.add($droplet.id)
    }
}

#If droplets get destroyed this should allow those numbers to get reused and not over increment
$findIncrement = 1
foreach($swarmID in $swarmIDs){
    $swarmNode = doctl compute droplet get $($swarmID) --output json | ConvertFrom-Json
    if($swarmNode.name -like "*$($findIncrement)*"){
        $findIncrement++
    }
}
#Sets the name of the droplet to be created
$newName = "swarm$($findIncrement).serverhobbyist.net"
#Run Ansible Playbook to Create Droplet
$command = "ansible-playbook CreateDroplet.yaml -e dropletName = $($newName)"
bash -c $command