Blockchain is our first step towards the Borg.
I have a set of business critical cron jobs that run overnight, once a week. If they don't run... it's bad news; really bad news. There's nothing worse than getting into the office, only to find out that one of the jobs threw an error and didn't complete. You get to a point where testing all the pieces in isolation just isn't good enough. I need to be confident that all of the pieces will work together. Time to get some integration tests in there (or feature specs, as we call them in the world of RSpec.)
A feature spec is a high level test that walks through an entire process to ensure that all of the pieces work together as expected. In a standard Rails application, that means using a headless browser to hit the UI, manipulate it in some way, and then check for an expected output. It touches all the layers; controller, model, view, and any other classes in between. It simulates a user initiating a process, interacting with an interface in a consistent, repeatable way, and expecting to get the same result each time.
What do you do when the thing initiating isn't a user? What if the UI isn't a web application? These might sound out of place for Rails, but if you're anything like me, you see them all the time. Crontab initiates multiple processes every day that run through a command line interface. These are rake tasks. I consider them features of my application, no different than the various features available through the web UI.
Uploading Files to Amazon S3 seems like something that should be relatively easy, given how common of a task it is. Oddly enough, I had to cobble a solution together with a mix of official and unofficial documentation, Stack Overflow answers, and GitHub issue comments. To make this easier for my future self and hopefully lend a hand to someone else in the same situation, I'm logging every piece of the puzzle required to be able to upload files to Amazon S3 using the Paperclip gem in a Rails app.
By now, I'm assuming that you've read Part 1. If you haven't, that's ok. Although this post follows thematically from Part 1, it can stand alone. Enough talk, let's get to it.
In Part 1 I talked a lot about this programming problem that I was asked to solve during a technical interview. Here's a rough reconstruction of that problem:
Use the BitTorrent encoding (Bencode) protocol to serialize the following data structures: String, Integer, List, Dictionary.
That's not the exact wording, but it's roughly equivalent. I suggest that you quickly review the Bencode protocol if you aren't familiar with it; just so that we'll be on the same page. I'll give a couple of examples from the Bencode wiki:
IN -> OUT1: "Spam" -> 4:spam
2: 3 -> i3e3: ["spam", "eggs"] -> l4:spam:4:eggse
4: {"cow" => "moo", "spam" => "eggs"} -> d3:cow3:moo4:spam4:eggse5: {"spam" => ["a", "b"]} -> d4:spaml1:a1:bee
When this problem was presented to me, I approached it from the perspective of OO imperative programming. Let me tell you: that got really messy, really fast.
Once upon a time I interviewed with one of the big SV startups. It was my first technical interview, and feeling understandably nervous about it, I did what I could to prepare myself. I read about other people's experiences in similar interviews, did some code katas, and solved some math problems. Mentally, I was prepared. From a technical perspective, I was missing something. At the time I was very comfortable with imperative programming, but hadn't yet been turned on to functional programming. I approached problems within a very imperative frame of thought. In most cases, that's not a detriment, unfortunately for me, in this case, it was.
I was confident that I could (eventually) solve any problem that they threw at me; the issue was that I didn't necessarily know how to solve it well. To know how to solve a problem well means being able to not only understand the expectations of problem, but also to be able to think in terms of the problem. How do you think in terms of a problem? I think it comes down to the idea that some problems lend themselves relatively better to certain kinds of solutions.
Let's start with a trivial example: calculating the sum of a list of numbers.
I've been reading a lot about functional programming over the last year or so. I started with Scala and more recently picked up Clojure. Of everything that I've read about functional programming, it seems that the focus is on immutable data structures, the different types of functions, and the different ways that functions can be used together. Generally speaking, there is mention of the difference between imperative and functional programming, but I haven't yet read something that has truly expressed how much of a shift in thought process it is to go from thinking imperatively to thinking functionally when programming. Maybe it's just me. Maybe I just have an imperative brain, or maybe 18 years of focused imperative programming has enframed my view of what it means to program.
When reading about Lisp, you'll often hear about the Archimedean moment when you finally understand that data is code and code is data. I really haven't heard a similar sentiment about finally understanding the shift from thinking imperatively to thinking functionally. Maybe it just comes more naturally to everyone else? Maybe people don't notice that the shift happens? Or maybe people don't think it's important?
For me, it felt unnatural, drastic, and one of the more impactful revelations in my programming career. When first learning about functional programming, I just devoured everything I could relating to it. Everything I read seemed to make perfect sense. It had an inherit beauty to it; composing functions felt like composing music. You're using these finite building blocks to create something akin to a perfect whole. All of a sudden it seemed only fitting that Haskell would have monads.
There's a heavy focus on list processing and recursion; you favour list traversals over iteration. You don't change data anymore, you transform it into new data. These are some of the highlights of functional programming. The problem is that it's possible to do all of that in a very imperative way. In the sense that you can essentially transliterate an imperative iterative solution into a functional recursive solution. You can write functional code that misses or ignores the shift in thought required for thinking functionally. Let's look at an example.
Plato's Theory of Forms is to Class Inheritance as Aristotle's Theory of Universals is to Object Composition