Yield Thought

it's not as hard as you think
formerly coderoom.wordpress.com

Last week I promised to share all the details about my cloud development setup - this is how I did it, in ten soul-crushing liberating steps:

Step 1: Get a Linode

I chose a Linode 512, which has been perfect for my needs. You get all phenomenal CPU power and an itty-bitty RAM. Surprisingly, this is enough as we’ll be doing most of our work on the command-line; you can always upgrade it later.

I also turned on the $5 a month backups. Zero-effort data safety for less than the price of a hot chocolate? Yes, please!

Step 2: Configure your Linode

Linode’s interfaces walks you through adding your new node. Pick a region close to yourself - you want to minimize the roundtrip time to the server. I spend most of my time in Munich, so I have mine in London and get a 30-40ms ping, which is great.

I also picked one of their default 32-bit distributions - if you’re not sure or don’t care then just choose the latest Ubuntu and you won’t go far wrong.

After a few minutes the machine will be installed and booted, ready for use! Time to log in and set things up!

Step 3: Build Vim

I found Ubuntu’s default vim didn’t have everything I wanted, but don’t worry! It’s easy to download and build it yourself:

install-vim

Step 4: Install lots of juicy plugins

I honestly can’t remember which plugins I installed and which ones I actually use. My currently-installed list is:

  1. acp.vim
  2. asynccommand.vim
  3. a.vim
  4. cctree.vim
  5. clang_complete.vim
  6. color_sample_pack.vim
  7. command-t.vim
  8. compview.vim
  9. cscope_maps.vim
  10. fuf.vim
  11. l9.vim
  12. NERD_tree.vim
  13. scratch.vim
  14. searchfold.vim
  15. taglist.vim
  16. tasklist.vim

You probably don’t want all of those; I wouldn’t be without command-t though. You can get them all from vim.org/scripts - you might want to install pathogen and get them that way; I just copied the existing .vim folder from my MacVim installation, hence all the cruft.

Note that command-t requires you to build a stub - follow the install instructions in command-t.vim and it just works.

To use the clang_complete plugin, you’ll need clang: “apt-get install clang” should do the trick. I had to play around with it a bit to get it working on my project, which included adding all the -I and -D command-line options to a .clang-complete file in the project directory.

Step 5: Create your .vimrc

Here’s my .vimrc - some of the things here relate to e.g. our internal tracker system and won’t be interesting for you, but it should be clear what most of these things do and which keys are bound to them:

.vimrc

Step 6: Configure GNU screen

I don’t recall where I got this from, but it works well for me:

.screenrc

You’ll also want to create a few aliases to connect to different screen sessions painlessly - my .bashrc contains:

.bashrc

You can probably change screen’s escape key in the .screenrc file too; I never bothered to look it up. Here, ‘s’ or ‘t’ will attach me to my work or personal sessions respectively.

Many people have recommended tmux - I haven’t tried it yet; screen hasn’t given me a reason to want anything else.

Step 7: Configure VNC

install-vnc

Feel free to pick a display that makes you happy but make sure it matches the DISPLAY=:5 line in your .bashrc!

Step 8: Get a super-sweet thin client

I chose an iPad2, (the 16Gb Wifi model). I haven’t tried working over (tethered) 3G yet, but ssh from my iPhone works well so perhaps that’s also an option if you’re often out of wireless contact.

If you already have a tablet that you like, use that instead - all you need is a good SSH client and a good VNC client. On the iPad I use iSSH and Jump for SSH and VNC respectively. I haven’t tried Prompt or Screens, if you already have them then let me know how well they work out.

Whatever you do, get an external keyboard. The Apple bluetooth one is amazing, I can thoroughly recommend it.

You’ll also want an adjustable stand / case for your tablet. Amazon offers plenty to choose from.

Step 9: Configure your thin client

The hardest part of setting up a SSH client on a tablet is getting your private SSH key on there without entrusting it to a third party. I split mine across multiple services and removed it after recombining it; maybe you don’t really care.

If you didn’t add the export TERM line into your .bashrc then remember to tell your SSH client to report its TERM type as “xterm-256color” - providing it supports this (iSSH does) then you’ll get to enjoy beautiful soft shades in your remote Vim sessions - joy!

To begin with I used iSSH’s VNC client, which worked perfectly well but was a pain to switch to quickly. Last week I started using Jump instead and love it, it’s a super-smooth experience with lots of cute little touches that make me smile. Swiping to it with the new iOS5 gestures is much easier than changing the view in iSSH and the next update will make it auto-reconnect after the 10 minute background timeout, making the whole experience seamless all the time.

I learned a lesson: take a bit of time and money and try out various different clients. You’ll be spending most of your working day in them; try to smooth out every aspect of the experience.

Step 10: Enjoy life in the cloud

There’ll be many other packages you want to install - Dropbox and Google AppEngine spring to mind - both can run in console-only modes.

If you’ve followed this far then you should have a fully-connected remote server and tablet client - welcome to the future!

Update: I’m still using this setup a year later - you can read on at iPad+Linode: 1 Year Later

On September 19th, I said goodbye to my trusty MacBook Pro and started developing exclusively on an iPad + Linode 512. This is the surprising story of a month spent working in the cloud.

It all started when I bought my first MacBook a couple of years ago. Frustrated by the inconsistent usage of ctrl/alt/option/arrow keys to jump words and screens and lines, I searched for a new IDE. Instead, I found Vim and fell in love. This isn’t another gushing post about Vim-oh-how-I-love-you-my-sweet-darling, but it’s important to the story - as we’ll see in a moment.

Although I like to use Python and GAE for my own projects, at work we write heavyweight C++/Qt code that runs on clusters such as the 200,000 processor Jaguar machine, so most of my time is spent in Linux and a lot of it on remote systems. Typically I’d develop in MacVim locally and run my code in VMWare Fusion or remotely.

One fateful day, VMWare and OS/X conspired to trash my shared filesystem, losing several days of uncommitted code in the process. I was angry.

While dd was recovering as much as it could, I started toying with the idea of giving up on local filesystems altogether. To my surprise, it seemed possible - even plausible.

I just had to try.

The Setup

It turns out you need a little more than just an iPad and a dream, but not too much more:

  1. iPad 2 (16Gb, WiFi)
  2. Apple wireless keyboard
  3. Stilgut adjustable angle stand/case
  4. iSSH
  5. Linode 512 running Ubuntu 11.04
  6. Apple VGA adapter

Total cost: around $800 + $20 per month

I Wandered Lonely as a Cloud

I typically start my day by catching up on the bug tracker chatter, mercurial diffs and other emails with the iPad in one hand while lying on the Combinat56 sofa.

I actually hate the Mail app for this; the stupid animation when archiving posts adds unnecessary delay and the archive button is uncomfortably placed at the top of the screen. More recently I’ve been scanning mails over IMAP with a python script instead.

Next, I lazily swipe to Safari and review my tickets for the day in our web-based bug tracker then return to the keyboard and fire off a couple of emails before settling back into coding - the new four-finger swipe gestures in iOS5 have really improved my life.

But we were talking about coding, which brings us back to Vim.

Vim: My Home from Home

Perhaps the only reason this transition has been so smooth was because my favourite editor / IDE looks and feels almost exactly the same running on an iSSH console as it did running locally on my Macbook. iSSH supports xterm-256color, which means you can still have pleasant colour schemes despite working in a terminal. All my plugins are there, my code-completion, quick navigation and so on.

In short, it’s a seamless transition from my MacVim envionment. If I were developing OS/X apps with Xcode, or used Eclipse or Visual Studio regularly this change would probably have killed me.

As it happens, working in the terminal on a remote Linode is even better than working locally, thanks to the magic of GNU screen.

GNU Screen is Magic

GNU screen is like a window manager for your terminal sessions, giving you multiple tabs, searchable history, activity/idle notifications and - best of all - persistence.

So I fire up iSSH, tap on my Linode connection and reconnect to the already running screen session. All my terminal tabs are exactly where I left them. Other SSH tunnels are still set up. My cursor is still in the same position. The clipboard is as I left it. It’s as if I never left, because for my side projects, I have a different screen session with a different set of tabs and editor instances running. Perfect separation.

It’s hard to overstate how pleasant it is to be able to return to exactly the same session each day; on a MacBook there’d usually be some other distracting programs left that I’d opened in the meantime, and of course any remote connections would have been dropped. At the very least I’d have used MacVim for something else in the evenings. It might be a largely psychological benefit, but it feels as if I can drop back into flow almost as easily as I left it.

The Good, the Bad and VNC

At work we develop a graphical parallel debugger, so I can’t spend all my time in the terminal. For hands-on tests and GUI work I need X. Luckily, iSSH has a workable, if not perfect, solution:

I tap on the VNC connection and up pops the GUI in shocking 1024x768 fidelity and a painful 256-colour palette. I could have 24bit colour, but somehow I prefer the extra speed to beauty. Although it’s a far cry from using a mouse to interact with a traditional GUI program, the on-screen ‘touchpad’ works better than I’d expect. And as it happens, being limited isn’t all that bad:

One good way to evaluate the usability of a program or dialog is to… try to use the mouse with just one finger.

Joel Spolsky

iSSH’s VNC isn’t quite as bad as pushing the mouse around with one finger, but it does make you consider users with lower screen resolutions, larger font sizes and mouse control that hasn’t been honed by countless years playing Quake and Starcraft. I’d be lying if I said I hadn’t wished for bluetooth mouse support some days.

Maybe It’s a Lifestyle

Today is not one of those days. I unwrap a chocolate croissant, make a fresh cup of tea and settle down to work; a quick hg pull -u && make to start the recompile, then ctrl-x to my editor tab and carry on coding while the rebuild happens. Ctrl-X is my screen’s ‘hotkey’; it defaults to Ctrl-A but on a wireless keyboard that leaves unicode characters in the terminal - I assume this is related to Apple’s support for some common Emacs key bindings in iOS. Strange, but easy enough to work around.

After a few minutes of coding the bar at the bottom of the screen notifies me that my compile has finished; I ctrl-x back, start a sanity test suite running just to be sure nothing horrible was broken overnight then carry on coding again. Compiling on the quad-processor Linode is around twice as fast as inside VMWare on my MacBook was. It’s also completely, blissfully silent. The only sound is the raindrop-like patter of the keyboard as I work. It doesn’t even get warm, a surprisingly refreshing change from the MacBook keyboard!

I swipe to the DuckDuckGo app and look something up in the Qt APIs, then swipe back. It’s becoming second nature, but I still miss a keyboard shortcut for task switching.

After an hour or two of uninterrupted development the UK team wake up and the first instant messages arrive. Thank heaven for the iOS 5 update! I use the imo messenger app, which is fine but of course before the notification center each pushed message would interrupt whatever I was doing with a frustrating popup. Now, the new notifications center behaves much more like a growl notification would; it lets me know and it gets back out of the way again.

I finish the function I was working on then four-finger swipe to the chat app; it’s my boss reminding me about our 11am conference call.

Skype-based conference calls work fine on both the iPad and the iPhone; at least, as well as VOIP calls ever work, i.e. fine after a few false starts while someone reconfigures their linux audio drivers. I appreciate not to having to think about that any more.

During the Skype call my iSSH session timed out in the background; it’s only held for 10 minutes or so. Fortunately a single tap reconnects me, and through the magic of gnu screen I’m back at exactly the place I left off again.

As is always the case, while fixing one bug I encounter another, apparently unrelated one. Time to submit a quick bug report - in the past I took screenshots on the iPad and emailed them to our tracking system, to get around not being able to attach in Safari. Now, I’ve switched to grabbing them and any log files or stack traces with a command-line app on the Linode itself. We use the Best Practical RT tracker (for our sins), which conveniently comes with a perl script to interact with it from the command line. Doing this is actually quicker and easier than uploading via a browser used to be; one command noting the ticket number and the attachments and I’m done.

The Cloud is Always With You

Lunchtime already! I close the iPad and head off with the others. The remote rebuild I left going will still be there; no need to worry about any uploads getting interrupted, it’s all happening in on the Linode, after all.

During lunch I pull out my iPhone and connect to the Linode again; damn, the build failed early on - out of disk space on one of our office machines. The iPhone keyboard is somewhat painful to use, but for rm -rf /tmp/build-2011-* it suffices. I kick off a new build while waiting for dessert to arrive.

Now I’m feeling nicely fed and just a little dozy; to keep awake I move to a standing workstation and set off some longer-running performance tests on a remote cluster. I find I move around between a lot of different working configurations with the iPad; much more frequently than I when using a laptop. I’m not sure why; perhaps it’s something about having a separate keyboard and screen that makes it easier to find a comfortable typing position, or perhaps it’s just the novelty.

The tests will take an hour or so; I close iSSH and review the latest spec / schedule changes my colleagues have made to an upcoming spec, stored in Google Docs.

Oh, Google Docs, why do you hate me so?

In a bitterly ironic twist, the only part of working in the cloud that doesn’t work smoothly is using Google’s cloud-based MS Office replacement. The mobile word processor is a joke, frequently losing the cursor position and sizeable chunks of text. The spreadsheet version is better, but is still a pathetic experience compared to the desktop version. Of course there are no native apps, and the desktop versions of the sites simply report script errors. I’ve even thought of trying to convince my team to switch back to OpenOffice.

Maybe it works better on an Android - no, wait, that’d be anti-competitive behavior and Google would never do something evil, right?

The tests have finally finished and the results are in; I move to the lounge area and connect up to the big HD TV with the VGA cable; something about seeing console-mode VIM in giant HD makes me smile every time.

There are some oddities in the logs; I copy and paste a few snippets to a colleague and discuss them in chat. I’m glad copy and paste work smoothly, although having 3 different sets of buffers (iPad, screen, vim) does lead to confusion from time to time. Idly, I wonder if any of my colleagues have noticed I’ve been using an iPad instead of a laptop for the last month.

6pm rolls around and it’s time to head home for the day. Despite a full day’s intensive use, my iPad’s still showing 15% battery life. I never bring a charger to work and I’ve never needed one. That in itself is a taste of freedom.

At the End of the Day…

Later that evening I pull the iPad out and open up Pages to finish a blog post. The house is quiet; only the distant sounds of the city drift in from outside and mingle with the gentle patter of keypresses on this delightful wireless keyboard.

I started this experiment because I fundamentally believe that most people don’t want to rearrange windows, babysit their own general purpose computers or back up their data. Sooner or later, almost everyone will work like this and I wanted a taste of what that might feel like. I expected to find something that didn’t work, but as the days turned into weeks and the weeks gathered into a month, I found I hadn’t returned to my laptop even once.

I don’t miss the weight. I don’t miss the keyboard getting warm when I’m compiling. I don’t miss its fragility, both physically and virtually. I don’t miss running out of power. To my surprise, I find I am happy. Coding in the cloud isn’t for everybody, but for my workflow it’s a perfect fit and I love it.

After a few minutes the perfect peace is rudely disturbed by the twin jet-engine that is my MacBook fans spinning up to full power on the shelf behind me. I can’t believe I used to put up with this all day and night. 

Despite that, I leave the MacBook running - it’s doing the only thing I still need it for on a regular basis.

It’s ripping DVDs.

Update: I’m still using this setup a year later - you can read on at iPad+Linode: 1 Year Later