imedo Development Blog

Stay tuned for follow-up posts and possibly a Cheat-Sheet to handle tricky scenarios. Comments are appreciated. Please send bug-reports and feedback for TinyMon to:

Popularity: 6% [?]

With Cucumber 0.4 the API for the output formatters has been changed. We updated our output formatter according to these changes. It is now compatible with Cucumber versions 0.4.x to 0.6.x.

The the updated code can be found here: imedo_ci_formatter.rb

Popularity: 1% [?]

We are proud to announce the immediate availability of the first version of css_doc. It is a tool to extract Javadoc-like documentation from CSS files. It was inspired by the work from the CSSDOC guys, but is NOT a complete implementation of their proposed standard. It is, however, quite similar, so that your existing CSSDOC documentation should probably work with css_doc.

This is the first release, so it is not what one would call “feature-complete”, but it is already quite stable at the moment. This version’s features include:

• File-level documentation for each CSS file in your project.
• Possibility to divide a single file into multiple sections.
• Rule-set-level documentation (a rule set is a set of selectors, separated by commas, together with their CSS properties).
• HTML-Code examples for the usage of your CSS rules are extracted. This is useful for building a style guide.
• Generates selector, section and file index pages.

For an example, please have a look at http://opensource.imedo.de/css_doc/index.html. The documented CSS file can be found here: http://opensource.imedo.de/stylesheets/style.css. I know the design of the css_doc documentation is not pretty, but we will improve on that in a future release.

If you like css_doc and are a Ruby hacker, or if you would like to improve the default design, please fork the project on github and send us a pull request. Of course the code contains lots of tests, but you are also welcome to add more tests, especially if you find bugs.

For installation instructions, please see http://github.com/imedo/css_doc.

Popularity: 7% [?]

For the impatient

Check out the demo application:
http://opensource.imedo.de/htmlmail

Install the plugin:
script/plugin install git://github.com/imedo/awesome_email.git

Learn how to use it below.

Introduction

Have you ever tried sending HTML emails to your users? If you did, you know for sure that it sucks big time: none of the usual ActionView helpers want to work, URL routing is disabled, layouts don’t work, and last but not least, the CSS you wrote for your email simply won’t work in any e-mail client except maybe Apple Mail. To solve all of the above problems, the awesome_email plugin comes to the rescue. Just install it into your vendor/plugins folder, and the rest comes by itself.
If you are interested in what works in which Email client check this link: A guide to css support in Email

What does it do?

There are a few interesting components in awesome_email:

• awesome_email adds layout support to emails. That means that you can use templates for e-mails just like you would with normal Rails Views.
• The HTML Mail’s CSS is automatcally inlined. That means that your designer and/or CSS guy can design the email in a web browser without worrying about how it might look like in excotic email clients. Yes, it works in Outlook, too, and no, it doesn’t work in Outlook 2007 without tweaking. The reason is a “stupid decision from Microsoft about Outlook 2007”, but we’re working on that one.
• ConvertEntities replaces Umlauts and other crazy symbols like ä, Ö etc. with their HTML Entitiy counterparts e.g. ä and so on.
• HelperMethods allow you to dump the content of the CSS file right into a style tag inside the header of your HTML mail.

How to use it

In your Mailer.delivery_xxx methods you can use

1
2

layout    "template_filename"
css       "css_filename"

to define which layout should be used and which css file should be used to create inline styles

CSS inlining

The cummulated style of each DOM element will be set as an style attribute when using css inlining.

Example:

your css file:

1
2
3

#some-id { font-size:2em; }
.some-class { color:red; }

your template:

1
2

<p id="some-id" class="some-class">Hello World!</p>

will result in the following code:

1
2

<p id="some-id" class="some-class" style="color:red; font-size:2em;">Hello World!</p>

Important!

Be sure to follow these simple conventions or otherwise awesome_emails’s magic will fail:

• The layout must be located inside app/views/layouts/{mailer_name}
• If you send mutlipart mails, check out the conventions on how to name your files: http://rails.rubyonrails.com/classes/ActionMailer/Base.html
  • So if you have these files inside of /app/views/{mailer_name}: signup_notification.text.plain.erb, signup_notification.text.html.erb ActionMailer will send a multipart mail with two parts: text/plain and text/html
• Your CSS file must be inside of /public/stylesheets

Dependencies

gems: rails 2.0.2, hpricot, csspool

Getting it, License and Patches

Get the complete source code through http://github.com/imedo/awesome_email. License is MIT. That means that you can do whatever you want with the software, as long as the copyright statement stays intact. Please be a kind open source citizen, and give back your patches and extensions. Just fork the code on Github, and after you’re done, send us a pull request. Thanks for your help!

ToDo

• More test coverage (as usual)
• make it more flexible with view paths
• rails 2.1 compatibility

Popularity: 100% [?]

In the past we had some serious issues with observers, that raised our environment loading time to ever higher ranges, which led me to take a closer look into why this happens.

I located the problem in the way the observe directive is used. You need to hand it a constant like User which directly leads to this constant being defined, i.e. the class loaded. That itself was no problem if the observers were loaded at some point where they are need and not on loading of the environment.

Models and Observers

Here I have to distinguish between where the model and where the observer are needed.

Model

The model, of course, is not needed untill you want to do something with it, like finding instances or creating new objects. Therefore you would not need to load the model class until someone called something on it, maybe hours after the environment was loaded. The observer implementation breaks this by requiring the constant as parameter (when you don’t use the observe method, then it will determine – and load – the class automagically).

Observer

The observer on the other hand needs to be there right from the start, since it must be ready to react on any change an observed model may make at any time. Therefore it is of course necessary to load observers along with the environment.

Postload Models

With these things figured out I tried to find a way to decouple the registration of the observer to the model from the model itself. So I created a plugin that will modify the way exactly this bit of code works. Observers will now register in one of two places. If the class is already loaded then it will work as usual, telling the class that it wants to observe the object’s changes. The magic now happens if the class is not yet loaded, because then the observer will just register with a hash, whose key is the class name, and the class itself will check this hash when being loaded for the first time. All observers in there will be hooked onto the class at load time – but at the model’s load time, not the observer’s.

I packaged up all this in a plugin called postload_models. You just need to put it into the plugins folder and it will start to work right away. It is backwards compatible, but you only get something out of it, if you don’t use the default way of defining observers:

1
2
3

  class MyObserver < ActiveRecord::Observer
    observe User, Post
  end

This will of course load the models User and Post since you hand it the constants (not just the names). Here the plugin kicks in and modifies the observe method to also accept Strings and Symbols.

The post-loading way would work like this:

1
2
3

  class MyObserver < ActiveRecord::Observer
    observe 'user_message', :post_subscription, "CommentRating", :FileCache
  end

That will make sure that none of User, Post, Comment or File will be loaded at this time. They may – of course – be loaded somewhere else before the observers are defined.

Popularity: 1% [?]

For the impatient

• Project Page
• Documentation
• Download
• Github repository

What is it?

Imagine you have some fancy acts_as_something plugin and, of course, you want to test it. How do you do it? In the Rails app you are developing it for? This could lead to longer environment load times and this is hardly testing in isolation. Creating a rails app just for testing the plugin? That’s better but creating a rails app for every new plugin – that’s so un-DRY! And you cannot test the plugin on its own. You could ship the bare rails app with your plugin but again: not DRY and lots of code you don’t really need for your plugin.

dry_plugin_test_helper to the rescue!

What it does

The gem sets up the test environment which means you have a stub rails app with your plugin and the following models:

• Article: belongs_to :author, has_many :comments
• Author: has_many :articles
• Comment: belongs_to :articles, belongs_to :user
• User: has_many :comments

The models will be added to a sqlite in memory database for fast testing.

To use the gem enter the following lines in your test_helper.rb or abstract_unit.rb (or however you may call it):

1
2
3
4

require 'rubygems'
require 'dry_plugin_test_helper'

PluginTestEnvironment.initialize_environment(File.dirname(__FILE__))

You can add your own test models using a migration in your test directory like
this:

1
2
3
4
5
6
7
8

 PluginTestEnvironment::Migration.setup do 

   create_table "animals", :force => true do |t|
     t.column "name", :string
     t.column "age",  :integer
   end

 end

If you don’t want the standard models you can initialize the test environment
like this:

PluginTestEnvironment.initialize_environment(File.dirname(__FILE__), :use_standard_migration => false)

How it works

The gem contains a stripped down rails directory structure and boots up this rails environment and then adds the plugin under test to this environment. It uses the sqlite3 in memory db for faster testing.

Caveats

You can really only test the plugin that you are testing. If your plugin has some dependencies to other plugins (why should it?) you cannot test it using this gem out of he box. You’d have to mess with the load path yourself (I haven’t tried it)

Todo

• support for RSpec and other testing frameworks (might already work – haven’t tried it)
• Automatic Rails version discovery
• Clean up / strip down rails env to bare minimum (It’s not completly clean yet)

Dependencies

The dry_plugin_test_helper gem depends on Rails and sqlite3-ruby

Getting it, License and Patches

Get the complete source code through Github. License is MIT. That means that you can do whatever you want with the software, as long as the copyright statement stays intact. Please be a kind open source citizen, and give back your patches and extensions. Just fork the code on Github, and after you’re done, send us a pull request. Thanks for your help!

• Project Page
• Documentation
• Download
• Github repository

Popularity: 1% [?]

Last month I had to write some stuff that affected our whole application and therefore I had to run the full test suite several times a day and wait for it to finish every time to find out what the errors are that it displays.
The output in the shape of “…E..F..FE..” is usefull to get an overview of the amount of errors but unless you wait til the end you can’t get a clue as to where the error occurred.

That bugged me quite a while and finally I wrote a plugin that shapes the output much better (at least to my mind).

When you use the TestOutputShaper you get this:

AlbumTest:    ....F.F...
AlbumControllerTest:    FF......FF...
PictureTest:    FFFFFFFFFF

At that point – without knowing the actual error – you can stop the suite and run the particular test to see what is broken.

When you work with small test suites that may not be a big win, but when you have a lot of code and have it well covered by tests, it can be a huge time saver.
Especially when you test for valid html your test suite can take like hours.

Popularity: 1% [?]

Recently, we released the background plugin, that allows you to run any Ruby code block in the background. We primarily use it in combination with ActiveMessaging, but any background processing framework like message queues, job queues or background tasks can be used with it.

Some people have asked us if we could add support to configuring the backend, such that it is possible to choose the queue in ActiveMessaging over which the block is sent. To keep it flexible for future releases, it is possible to configure any backend with an options hash like this:

background :handler => [ { :active_messaging => { :queue => :my_queue } },
                         { :disk => { :directory => '/tmp/my_queue' } },
                           :forget ] do
  # do your task
end

Granted that the example above is a little verbose, but usually you wouldn’t have to configure that much. However, if you do, you could wrap the example above like this:

def my_queue(&block)
  background :handler => [ { :active_messaging => { :queue => :my_queue } },
                           { :disk => { :directory => '/tmp/my_queue' } },
                             :forget ] do
    yield
  end
end

Note that the :fallback option of the last version is no longer available; just give an array to the :handler option.

Getting it, License and Patches

Get the complete source code through Github. License is MIT. That means that you can do whatever you want with the software, as long as the copyright statement stays intact. Please be a kind open source citizen, and give back your patches and extensions. Just fork the code on Github, and after you’re done, send us a pull request. Thanks for your help!

• Project Page
• Documentation
• Download
• Github repository

Popularity: 3% [?]

Update

There is a new version available. More details are available here.

For the impatient

• Project Page
• Documentation
• Download
• Github repository

Introduction

Every Rails developer knows this: Your application is fast and responsive, up until a point where the data set handled in one request gets so large that request times become unacceptable. The obvious solution is to identify code that does not have to run immediately, but can be delegated to another process. There are several ready-made solutions for delegating execution of code to a background process, like ActiveMessaging using ActiveMQ, BackgrounDRb using a DRb server, or databased-driven Job queues.

The problems with each of the above mentioned approaches are:

• Responsibilities get cut out of objects (most of the time, the code running in the background process actually belongs to an object residing in the foreground process). This can be solved by delegating the time-consuming task to the background process, which clones the object, and delegates the task back to it, only in another process.
• Not DRY. Background processes tend to repeat code, like the above mentioned back-delegation. This can be reduced to a certain extent, but there will always be some overhead.
• Not Failsafe. Almost always, the task at hand is delegated over a socket of some sort. If the other process is busy, hangs or is down, there can be timeouts which result in ugly exceptions, and in the end, discard the task. To make background processing failsafe, you need to write a lot more code, which is repetitive as well.

We want to present our solution to all of the above problems, that also is very elegant. Running a task in the background is as easy as

background do
  # run your code
end

The communication with the background process is configurable, as is the error handling. For example, you could use ActiveMQ for queueing your background tasks. If the connection to ActiveMQ fails for some reason, the task could be executed in-process. If this fails (e.g. because of an error or timeout), the task would be dumped to disk for a later replay when the problem is fixed.

How does it work?

Actually, the solution is pretty easy. The block’s code and local variables needed by the block are serialized and sent to the handler, which then evaluates the block in the context of the local variables. The attentive reader might notice that it is impossible to serialize code blocks, let alone know all the block’s local variables in advance. Well, that is almost true.

There is a genius piece of code, called proc_source, that allows you to serialize code blocks. It works by parsing the source file that contains the block. This is possible, because code blocks know where they are in the source code.

It turns out that the local variables can’t be accessed with plain Ruby. But that is actually a good thing, because we might want to control which objects are sent over the wire, and only choose the ones that are actually needed, in order to save computation time and bandwidth.

So, to attach local variables to the code block, you’d use the following method call:

background :locals => { :user => current_user } do
  user.do_some_time_consuming_operation
end

Failsafe background processing

To make sure that your task is executed even when your background process of choice is not available, you can specify a handler, and a couple of fallback handlers. The handler and fallback handlers are tried in order, until the first one succeeds.

background :locals => { :user => current_user },
                  :handler => :active_messaging,
                  :fallback => [:in_process, :disk, :forget] do
  user.do_some_time_consuming_operation
end

In this example, the :active_messaging handler is tried first. If it fails, the code is executed in-process. If it still fails, the code is dumped into a file, and if even this fails, the task is discarded.

The self object

One of the amazing things is that you can use the self keyword inside blocks. This works because the object, in which the code is executed, is serialized as well.

class User
  def some_operation
    variable = some_evaluation
    background :locals => { :variable => variable } do
      self.do_something_with(variable)
    end
  end
end

Error reporting

As a developer, you might want to be informed when something goes wrong, in order to fix it. But since every project uses a different error reporting system, the error reporting is configurable. Also, some code is rather important, while other code is optional, so that you might not want to be informed about every error. To specify the error reporting, use the optional :reporter parameter:

background :reporter => :exception_notification do
  # background code
end

Note that errors are only reported if an exception occurs while talking to the background process; if you want to be informed when an error occurs while the block is executed in the other process, you need to implement your own reporting for the backgrond process.

Decorating existing methods

Most of the time, you want a whole method to be executed in another process. To make this pattern DRY, the background method can be used as a method decorator, when it is called in class-level scope:

class User
  def do_something_complicated(parameter, argument)
    # complicated things
  end

  # execute all calls to do_something_complicated in the background
  background :do_something_complicated, :params => ['parameter', 'argument']
end

Note that you have to specify all of the methods parameter names as they are written in the original method’s definiton. This is not 100% DRY, but neccessary to correctly send all the parameters to the background process. If you have a suggestion on how to avoid this, please let us know.

Default configuration

You can configure the default background handler, a default fallback chain as well as a default error reporter. The configuration lies in the Background::Config class.

Security Issues

As with any background processing, you need to be careful about the requests that are processed. Since the background plugin executes arbitrary Ruby code, you need to take special care that no unfiltered user input is injected. Make sure that your firewall does not allow connections from the outside, and that the code that connects from the inside is controlled by you. We don’t take any responsibility for any damage caused by the operation of the background plugin.

Limitations

Since singleton objects can not be serialized, all of the singleton methods are stripped away before objects are sent to the background process. Be aware of this fact if you rely on these methods. Most of the time, it should be easy to extend the objects again inside the code block.

Dependencies

The background plugin depends only on ActiveSupport, which is part of Rails.

Getting it, License and Patches

Get the complete source code through Github. License is MIT. That means that you can do whatever you want with the software, as long as the copyright statement stays intact. Please be a kind open source citizen, and give back your patches and extensions. Just fork the code on Github, and after you’re done, send us a pull request. Thanks for your help!

• Project Page
• Documentation
• Download
• Github repository

Popularity: 31% [?]

We are proud to announce the first alpha release of our unobstrusive user interface framework, written in Javascript. This release is codenamed THC2, and will be renamed as soon as we find time to think about a better name. Suggestions are always welcome.

What is Unobtrusive Javascript?

The idea behind unobstrusive Javascript is to decouple the Javascript from the HTML-Code, similar to decoupling style information from HTML. There are lots of immediate advantages in this approach:

• Optical aspects; it just looks nicer.
• Its DRYer. Using inline Javascript makes you repeat code all over the place.
• Easier to maintain validity. Using inline Javascript invites you to do certain hacks that break XHTML compliance.
• Bandwidth savings. No Javascript in the markup means less markup.
• Web designers understand your markup. Heck, they even start to understand the whole thing, and deliver Javascript-enriched interfaces—without using any Javascript.
• It’s compatible with screen readers.
• In contrast to inline Javascript, unobstrusive Javascript can be completely unit tested.

Because of our somewhat specialized audience, we have to present interfaces that are understandable by accessibility helpers, more than anyone else. Invalid XHTML, inline Javascript, seemingly unneccessary markup and so on, all might get in the way of accessibility software. So the main design goal for this Javascript library seems a bit paradox: Work with Javascript not working or turned off. This is actually another nice property that can be achieved using unobstrusive Javascript: Just design an interface with minimal markup that is usable without Javascript, and then enrich it with Javascript afterwards. Unobstrusive Javascript is all about enriching interfaces.

Features

• The library features a total of 12 helper classes, 18 ready-to-use widgets, and 6 extensible base classes to more complex widgets. For details, see the Demo page or the Documentation.
• The widgets are extremely easy to use, and the documentation is almost complete.
• There is a cross-browser javascript profiler, written in javascript, as an extra, available here.
• The library is unit-tested, so there is nothing to be worried about (some tests are still missing)

What about an example?

Ok. Suppose you need a link to open another page in a popup window; not a target="_blank" window, no, a real javascripted popup window. The easy and obvious way to do this is the following:

<a href="#"
  onclick="window.open('http://www.wikipedia.org', 'popup', 'width=640,height=480');
    return false;">Click</a>

Or, as a link: Click

The first problem of this solution is that Javascript does not belong in HTML code. It simply doesn’t, for the above mentioned reasons. The second problem is that on failure, no window is opened at all. Failure might happen, when Javascript is turned off, or a screen reader is used.

<a href="http://www.wikipedia.org"
  target="_blank"
  onclick="window.open('http://www.wikipedia.org', 'popup', 'width=640,height=480');
    return false;">Click</a>

Or, as a link: Click

This is an improvement to the first version, in that the actual target URL is given in the href attribute, so this has a real chance to work with screen readers. Also, for turned-off Javascript, the URL opens in a new window (note the target="_blank" fallback). There are two problems with this code, though. First, it is not DRY: The URL is repeated. Also, several occurrences of this kind of popup link repeat almost identical Javascript. This is a waste of bandwidth. Second, it is hard to understand. Try to have your CSS guy debug HTML code with inline Javascript. It isn’t fun to watch, trust me.

<a href="http://www.wikipedia.org" target="_blank" class="thc2-popup">Click</a>

Or, as a link: Click

This last version solves all above mentioned problems. First, it uses the target="_blank" fallback to open a window in any situation. Second, it has the target URL in the href attribute, so it works with accessibility tools. Third, and most important, it contains no trace of Javascript.

So, how does the link know that it has to open the link as a popup window? The answer is in the class attribute: The class thc2-popup tells the THC2 Javascript framework to apply the popup behaviour on the link. The popup behaviour, or PopupWidget, installs an onclick event handler on the link, which opens the URL that is specified in the elements href attribute in a popup window.

The code for the PopupWidget class might look something like the following:

var PopupWidget = Class.create(Widget, {
  initialize: function(element) {
    Widget.prototype.initialize.apply(this, arguments);
    this.url = this.element.href;
    Event.observe(this.element, "click", this.showPopup.bindAsEventListener(this));
  },

  showPopup: function(event) {
    event.stop();
    win = window.open(this.url, 'popup', "width=640,height=480");
  }
});
CurrentPage.registerBehaviour("thc2-popup", PopupWidget);

On first sight, this doesn’t look like any bandwidth saving at all. You’re almost right. Except for the fact that Javascript classes can and should be put in an external file, which can be cached by the browser forever. So over time, you will actually save bandwidth.

Dependencies

The THC2 framework depends on the Prototype (Version 1.6 and above) and Scriptaculous (version 1.8 or later) libraries. A few optional widgets need the TinyMCE WYSIWYG editor (version 3 or later); without TinyMCE, the widgets will use regular text areas.

Getting it, License and Patches

Get the library or the complete source code through the links on top of this page. License is MIT. That means that you can do whatever you want with the software, as long as the copyright statement stays intact. Please be a kind open source citizen, and give back your patches and extensions. Just fork the code on Github, and after you’re done, send us a pull request. Thanks for your help!

• Demo page
• Documentation
• Download
• Github repository

Popularity: 2% [?]