Mind shards

I’ve been using a Chromebook lately as my primary machine. In my first post I’ve explained the basic setup and issues I was having. Having used my Chromebook for a while now I think I have (even) more sensible things to say.

VPS

I think I have finally settled on a VPS. My ansible configs have come a long way. With regard to RAM: 2GB is enough, 1 GB is not, it’s that simple. It also matters which virtualization you use. I’ve started out with OpenVZ, this is what all the cheap VPS’s are using. The major downside it that you are unable to run kernel modules with it.

FUSE

And I wanted to use a kernel module, FUSE. This is used to mount external filesystems. I wanted a ‘drive’ back by one of the big storage providers. This drive I could take with me as a moved from VPS to VPS. After much deliberation I settled with S3QL. This requires FUSE (as they all do). This meant choosing any other virtualization over OpenVZ. Currently I’m running a KVM VPS at FileMedia after trying the 1GB offering at DigitalOcean.

Not settled yet

S3QL is nice, it encrypts my stuff on the backend but that means I can only access it via S3QL. Bittorrent just released their Sync offering and I tried it out using my RaspberryPi as a store, this seems to work well and I plan to start using it.

Music

This is still hard. I’ve sort of settled on Grooveshark and, surprisingly enough, my phone which I use anyway daily for tunes.

Overall

I like my Chromebook, however it has some distinct disadvantages. For one it is slow. It’s not that I have to wait a lot but it is just not as snappy as my old Macbook (not that I expected it to be so). I would love for it to be just a tad more powerful. The other downside is external screens; while they nearly always work, disconnecting always leads to a frozen Chromebook. I now just power off the device, booting is quick enough. Seems that after 20 years external monitors are still hard. Then there is printing and scanning. I still have some hope for printing but scanning seems just not possible. My Canon LiDE 110, a devise I absolutely adore, will sit idle. My aging Lexmark E120n network printer will also gather dust.

To end on a happier note; battery life is just great. It is also perfectly predictable, all the compiling I used to do now happens on my VPS!

Update followup: One month in

I drank the cool-aid and bought a Samsung ARM Chromebook. My Macbook Pro needed repairing and I can’t go without a laptop for two weeks. For a living I am a Ruby on Rails developer, most of my time I spent on either the commandline and Vim or in a browser. I figured I just as well might get a Chromebook which only has these two things. My plan was to use plain Chrome OS with a VPS. I really didn’t want to install Ubuntu on the laptop. In the end the only thing I tweaked on the Chromebook itself was to run it in developer mode.

VPS

Recently I have gotten quite enthusiastic about the devops tool Ansible which I now use to provision my VPS. I decided to do this not only because I liked Ansible but also because there are so many different VPS vendors out there, I wanted to be able to quickly and painlessly switch from one vendor to the other. Currently I rent a 2GB VPS from BudgetVM for $10 a month. Initially I was looking for a 4GB VPS but the price point was more important to me and who knew how my RAM I would need? After having run this setup for a couple of days I think 2GB is more than enough, actually I now think the order of importance is: latency, disk IO, CPU, RAM. What helped me enormously was Serverbear.com, this site finally gave me insight in the overgrown VPS market (and they have coupons too!).

Ruby

Many people are compiling Ruby from source for their servers. I think that is silly to say the least. Initially I set out to create my own Ubuntu Ruby packages, this is not for the faint of heart and I was struggling. Then I found Brightbox, bless them! These guys are maintaining a PPA with the latest and greatest Ruby builds (also an experimental PPA). They even threw in some optimizations. This saved me hours.

Development

Shell

Lots of road bumps! I have tried ‘crosh’ the native shell of Chrome OS and I have tried Secure Shell. The main downside of crosh is the it runs in a tab which then eats your Ctrl-W. I really need that. I have twiddled with vim bindings and I might get back to it but for now I have settled for Secure Shell. The latter one makes it easier to store your ssh keys, which, by the way, I had to generate on a different machine (unless you run in Chrome OS developer mode).

Windows

I also struggled with a setup in which I could use my windows effectively. On my Macbook I used iTerm which had several tabs. Running Secure Shell in a tab is not possible which it eating Ctrl-W so I run that in a window. For a tabs substitute I now use tmux, I’m still getting used to it. So now I only have 2 windows between which I can cycle with ‘alt-tab’. One running Chrome and the other running Secure Shell. I find it very important that with one particular keystroke I always get to the same spot.

Downsides

Music

I still haven’t found a proper way of playing my tunes. I have a large iTunes library but I would be willing to switch to some online service for less than $10 per month. For now I resort to online radio which involves Flash players. These are no fun at all and cause the load on my Chromebook to steadily rise to around 3.5 at which point things start to slow down noticeably.

Downloads

I receive email, sometimes even with an attachment. This I can download but what to do with a .xsd file? What I would really like is for my Downloads folder to be mounted on my VPS.

My resources

Provisioning

In my vps-ansible-config repo all provisioning comes together. It does:

• install Ruby
• add new user
• install basic packages
• fixes locale
• sets up my dev environment
• links up my dotfiles

Dotfiles

I have seriously spent time on my dotfiles. I have files for:

• vim
• zsh
• git
• tmux

Last week I found a piece of code which put expectations (for example should_receive or the likes) in before filters in your spec file. I believe this is not correct in subtle ways.

• A before filter is meant to define the ‘world’ your RSpec is going to run in. Not to set expectations. These go into it blocks.
• Besides that, when putting expectations in a before filter, the expectation is needlessly checked ever it block. Imagine the expectation not being met. All of the sudden all your specs turn red. I’d rather see a specific spec fail so I immediately know where to look.

So I’m suggesting to put no expectations in your before filters and if you feel the urge to do so it probably needs it’s own it block.

An example of how it should be done:

describe "foo" do
    before do
        some_object.stub(:some_method).and_return(:some_value)
    end

    it "manipulates the object" do
        some_object.should_receive(:some_method).with(:some_argument)
        some_invocation
    end
end

Recently I found myself playing with both Go and a Raspberry Pi. I’ve build a some Go programs on my Mac but copying over the executable to the Pi did not work (surprise!). It is actually fairly easy to crosscompile a Go binary:

GOARM=5 GOARCH=arm GOOS=linux go build your_program.go

It’s all pretty self explanatory, the one thing that confused me was the GOARM variable. I’d expect the GOARCH to be enough. But apparently there are different ARM types out there. A full list can be found on the Go site under Optional Environment Variables.

Sometimes you have a binary which insists on asking you questions when installing it. I ran into that with the linux Crashplan installer.

Of course there is a solution to this, called ‘here documents’. This allows you to issue commands to a script or program by piping in a piece of pre formatted text. Usually the CR/enter/return is represented as a just that. Create the here document with enters at the appropriate places.

With an ansible script that is not possible. Luckily you can use \n in these cases: ansible hostname -i ~/ansible_hosts -m shell -a "/some_path/install.sh <<EOF\nanswer to question 1\nanswer to question 2\nEOF" Not that there are 3 \n’s. You need an additional one after the first EOF.

I am going to return my Play:3. This saddens me a great deal and I would like to explain why I’m doing so.

First a little bit about my setup. My wife and I both have an (Android) smartphone with music. We also both have a Macbook with iTunes. Hers with a small library, mine 80GB. We have a Sonos Play:3 and a bridge.

First of all the good parts; - I love the setup. Easy, simple, clean. Great, great work. - I also love the sound the Play:3 produces. I couldn’t believe it when I heard it in the shop. So small, such great sound. - The ability to control the music with any device. Whether it was streaming from my laptop or hers. This is how it should be.

But there are, sadly, still enough reasons to return it.

1. The controller is build for a touchscreen. Which leads to a subpar experience on the desktop client. For example, pressing spacebar won’t pause/start the music. Clicking on the volume bar (upper left) won’t set the volume to where I clicked. It just takes a tiny step. Stuff easily fixed, I’m sure, but disappointing none the less.

2. This is really unacceptable; the podcasts I view in iTunes are mixed in my Music Library. I can’t believe it. Import podcast for all I care but don’t mix them in my Music Library.

3. When I add music to my iTunes library (which I do regularly) I have to wait till the Sonos controller decides to sync. What I’ve done so far is to go to Preferences -> Music Library -> Advanced and set the automatic update to 1 minute in the future. Not even a button ‘sync now’. Or better yet, just keep an eye on the file system (inotify anyone?).

4. With two libraries on two laptops it happens often that one of them is offline. Why does the controller let me select a track which is offline? It really isn’t that hard to know if a share is up.

5. As mentioned earlier my wife and I added both our Libraries to Sonos. Our libraries overlap. What happens now when her laptop is offline is that I can’t play songs which I know I have on my disk. Sonos thinks the tracks are located solely on her laptop, which is not true. I can go to Music Library -> Folders and select it there.

6. Playback from an Android device is nigh impossible. doubleTwist sorta does it, Twonky is barely capable. Playto is bad, as is iMediashare. Besides, all these apps are large (~20MB) which makes them appropriate only for mid-high to high end smartphones.

7. Would it have killed anyone to install a line-in on the Play:3? I feel this is only done to up sell to the Play:5. (That almost worked BTW.)

8. When a friend of mine came around last week he wanted to let me listen to a great new band he discovered. Without a line in I mounted his phone on my computer. I then proceeded to import his songs in my iTunes library, I then went on to the Sonos preferences and did the ‘1 minute in the future’ trick. Waited far more that 1 minute. In that time I had ample time to explain to him why I bought a device of €300 which could to this simple task.

This, in a nutshell, is why I’m (seriously considering) returning my unit.

Lots of people still use Chef for their devops needs. I briefly used Chef about 6 months ago and got very frustrated. I find the process convoluted and confusing. For our Riak cluster we needed a deploy script to move an install script and resources to the remove host. The install script installed Ruby, some required packages only then the actual Chef recipe was run. Maybe I was doing it wrong, but then I’m of the opinion that the whole thing is too complicated.

Ansible

With Ansible you can:

• Run one-off commands (want to know the loadon your cluster? ansible riak-nodes -m 'w'
• Provision servers with a simple YAML file
• Even replace Capistrano

Ansible is opinionated, questions I have asked on the mailing list were often quickly answered with a “you’re doing it wrong”. For example:

• Adding lines to a configuration file is considered smell. This leads to a remote system of which the state is not clearly defined.
• Downloading packages from a remote source? You can’t be sure this source is still up. Download the package yourself and copy it over.
• Downloading a tarball to compile it on the spot? Do it once and roll a package.

A “playbook” is the Ansible equivalent of a cookbook in Chef. It is a play YAML file:

God bless it’s simplicity. (Yes I know the playbook is still a work in progress)

RSpec did it again. It blew my mind. Just when I was doubting how awesome it was.

At Skylines we run a lot of services. API’s, websites, apps. You name it. When something goes down we want to know about it. Not when a client calls but the second after it breaks.

As a good developer you write a bunch of (unit/integration) tests. These tests are used when writing the code. But when the code is deployed all bets are off.

Our applications have a lot of moving parts and when one breaks that impacts our user experience. We want to monitor the experience our users are getting. It our API still up? Does it return valid JSON? Contains the JSON returned the fields we expect it to hold?

These questions sounded eerily like your run of the mill RSpec tests. I saw two problems with RSpec. Unlike regular tests we wanted to run tests continuously on live applications. Also RSpec just outputs on the console whether or not a test failed, we want to receive emails/texts when stuff breaks.

Turns out these aren’t problems at all! Years ago David wrote about the metadata properties you could pass to a describe or it method. Turns out this is ideally suited for what we want. Flag an example (or example group) as being a live test:

describe "live tests", :run_live do
  it "should return a 200 status code" do
    response = Curl::Easy.http_get("http://google.com")
    example.metadata[:live][:time_to_first_byte] = response.start_transfer_time
    response.header_str.should =~ /200 OK/
  end

  it "should return valid JSON"
  it "should have some fields in the JSON"
end

And in spec_helper.rb:

config.after(:each, :run_live => true) do |example_group|
  #do whatever you like with the exception and possible metadata
  example_group.example.exception
  example_group.example.metadata[:live]
end

All we need now is a daemon on some random server which checks out every repo we have and periodically run:

$ rspec -t run_live

Happy.

Configuration management

Cdist is a little piece of software which makes setting up new servers a little less painful. I believe the main advantage of using software like this is not the easy setup but the fact that the process creates executable documentation. Cdist is new when compared to Puppet, Chef and a couple of others. It appealed to me because it’s not much more that a glorified bunch of shell scripts. The other approaches I saw required a central server, pushing code around and not the mention the requirement that Ruby should already be installed on the target system.

Getting started

This couldn’t be easier, just clone the repo. You’ll need Python3, I suggest using Homebrew instead of Mac Ports. It took me a bit more time to understand that all my code should go in this cloned repo. Just start typing away. The maintainer suggests pushing to a different remote and to work on your own branch. This way a git pull will bring you up to date. A valid strategy I suppose but uncommon.

But then…

I followed the quickstart example but as you notice it’s a link to the git repo as this is not yet officially released. Following along is pretty easy until I hit

$ echo '__file /etc/cdist-configured' > conf/manifest/init

What does that do?

Types

The __file denotes a ‘type’, the file immediately behind it is the argument you might pass. These two combined form an object which can be reference later. I’ll get back to that (under Dependencies).

Manifest

The conf/manifest/init is the most important file. As far as I understood this file contains a long case statement enumerating all your servers. I’m just hoping you can use regular expressions or groups, all the nodes in our Riak cluster have the same setup and I’d hate it to list them all.

So

$ echo '__file /etc/cdist-configured' > conf/manifest/init

will make a manifest which does little. The __file type will create a file if it is not there already.

Do it already!

Assuming you can login with your SSH key as root:

$ cdist config your-remote-server

I was momentarily confused by the config switch as there also is an install switch which function I have yet to determine. The above command will go to your server and configures (or installs?) it.

Dependencies

More often than not you want to execute some piece of code if a condition is fulfilled. Cdist has a special feature to declare dependancies. You can assign a value to the variable require immediately follow but the code to be executed when the condition holds:

require="__file/etc/cdist-configured" __link /tmp/cdist-testfile --source /etc/cdist-configured  --type symbolic

This seems weird and I think it is. The advantage of doing it like this is that you can use the magic objects created when you call a type.

EC2

As a small side note; Cdist’s OS detection does not work on Amazon’s standard AMI’s. This bit me when wanting a quick, clean server to try this out on.

It is official; I’m looking for a job in the Utrecht or Amsterdam area. The position I’m looking for combines cutting edge technology with a healthy dose of pragmatism and a great team.

After selling my startup company I’m currently very much interested in Big Data and AI (I even have a degree in the latter!). I can actively participate in, or initiate, the development of a business or product by providing realistic technical insight, the necessary programming skills and the ability to motivate the guys (gals?) implementing the idea.

For a professional profile check Linkedin, for a more personal view check Twitter or this Tumblr.

If you think I can help you, please do email me on harmaarts@gmail.com.