imedo Development Blog

In the past we had some serious issues with observers, that raised our environment loading time to ever higher ranges, which led me to take a closer look into why this happens.

I located the problem in the way the observe directive is used. You need to hand it a constant like User which directly leads to this constant being defined, i.e. the class loaded. That itself was no problem if the observers were loaded at some point where they are need and not on loading of the environment.

Models and Observers

Here I have to distinguish between where the model and where the observer are needed.

Model

The model, of course, is not needed untill you want to do something with it, like finding instances or creating new objects. Therefore you would not need to load the model class until someone called something on it, maybe hours after the environment was loaded. The observer implementation breaks this by requiring the constant as parameter (when you don’t use the observe method, then it will determine – and load – the class automagically).

Observer

The observer on the other hand needs to be there right from the start, since it must be ready to react on any change an observed model may make at any time. Therefore it is of course necessary to load observers along with the environment.

Postload Models

With these things figured out I tried to find a way to decouple the registration of the observer to the model from the model itself. So I created a plugin that will modify the way exactly this bit of code works. Observers will now register in one of two places. If the class is already loaded then it will work as usual, telling the class that it wants to observe the object’s changes. The magic now happens if the class is not yet loaded, because then the observer will just register with a hash, whose key is the class name, and the class itself will check this hash when being loaded for the first time. All observers in there will be hooked onto the class at load time – but at the model’s load time, not the observer’s.

I packaged up all this in a plugin called postload_models. You just need to put it into the plugins folder and it will start to work right away. It is backwards compatible, but you only get something out of it, if you don’t use the default way of defining observers:

1
2
3

  class MyObserver < ActiveRecord::Observer
    observe User, Post
  end

This will of course load the models User and Post since you hand it the constants (not just the names). Here the plugin kicks in and modifies the observe method to also accept Strings and Symbols.

The post-loading way would work like this:

1
2
3

  class MyObserver < ActiveRecord::Observer
    observe 'user_message', :post_subscription, "CommentRating", :FileCache
  end

That will make sure that none of User, Post, Comment or File will be loaded at this time. They may – of course – be loaded somewhere else before the observers are defined.

Popularity: 1% [?]

I want to say a few things about a I read lately, called Design Patterns in Ruby.

In this book, Russ Olsen discusses 14 out of the original 23 design patterns described by the GoF. In addition he also talks about 3 not unique but very common patterns in Ruby. Alongside he gives a very nice overview of Ruby for the Ruby beginner and shows some sweet little tricks.

I find the book very comprehensive with the special paragraphs of when to not use a certain pattern, which is really helpful to not get overly enthusiastic with a pattern and try to use it everywhere. Design Patterns are after all solutions that you should use when you found a fitting problem and not when you made a problem fit.

Eventually I created a presentation with a short (not so short, actually) overview of the patterns for my colleagues which I want to share with all of you here.

The presentation on video:

Online Videos by Veoh.com

The slides:

Special Thanks

to Jonathan Weiss, who’s introduction and review pointed me to this book.

Popularity: 1% [?]

Last month I had to write some stuff that affected our whole application and therefore I had to run the full test suite several times a day and wait for it to finish every time to find out what the errors are that it displays.
The output in the shape of “…E..F..FE..” is usefull to get an overview of the amount of errors but unless you wait til the end you can’t get a clue as to where the error occurred.

That bugged me quite a while and finally I wrote a plugin that shapes the output much better (at least to my mind).

When you use the TestOutputShaper you get this:

AlbumTest:    ....F.F...
AlbumControllerTest:    FF......FF...
PictureTest:    FFFFFFFFFF

At that point – without knowing the actual error – you can stop the suite and run the particular test to see what is broken.

When you work with small test suites that may not be a big win, but when you have a lot of code and have it well covered by tests, it can be a huge time saver.
Especially when you test for valid html your test suite can take like hours.

Popularity: 1% [?]

Rescue Considered Harmful

That is a pretty crude statement, how do I come to this?

How come I say that?

Every now and then you run into issues where you code behaves in a way you don’t understand. Then the it’s high time to review the code.

Yesterday I worked on such a bug where I just didn’t see the error in the code and without any exception the controller did just respond nothing to the request. Weird stuff, I thougt.

So I slept a night about that problem and today called Marcel to look it over together with me. What we eventually found was this snippet:

1
2
3

  rescue
    render :nothing => true
  end

at the end of the method, which I overlooked totally last night. And at first sight you see this as a source of confusion. rescue is eating up all exception output, directing it to nowhere (aka. /dev/null).

So I wondered why would someone put that into code?

Answer is simple. To quote pragprog’s Venkat and Andy: “Quick Fixes Become Quicksand” (from Practices of an Agile Developer Tip #2).

Raising exceptions there messed up some JS selectbox when you selected “nothing”. Therefore the exception got rescued in a quick fix. Problem with that is that it never got send anywhere. Simply putting the exception into some mail does the same job now but we do get notified when something goes wrong in that method.

Conclusively

I should alter the initial statement a bit:

Rescue Without Handling the Exception Considered Harmful

Popularity: 1% [?]

For several actions in our setup we use a message queue, namely activemq to queue up processes, like, say, write operations to the DB server, or some not time critical stuff.

This works very well in general except for the fact that the poller who reads from the queue is blocked as long as the current action takes to finish.

One of the things we queue this way is a tool for heavy operations, like sending the newsletter to all our users. Of course that’s one message in the queue telling the process all the users to execute on, but eventually there will be one action performed for every single one – easily blocking the poller for hours depending on what is done.

So tonight I sat down and figured out a way to decouple this particular processor (that’s what we call them) from the general poller. The method I found best was to use script/runner, but at first gance I didn’t know how to pass in all the params on the command line (because the ‘param’ is a not exactly small hash dumped as yaml). Then I remembered the bash heredoc syntax which looks just like ruby’s and works quite the same way.

1
2
3
4
5
6
7
8
9
10
11

script/runner ProcessRunner.run <<EOF
---
source: 1
targets:
- 4
- 6
- 7
- 15
action: deliver_group_newsletter
object: Group_16
EOF

This works quite fine inside a shell so I just tried it with exec – and it worked, though exec is not the right choice there, since it does decouple the execution from the surrounding process but by replacing that process with what you pass to exec. So the poller died after every execution.

Next I tried system() to execute the runner but then again, system() waits for the process it runs to finish, what makes just no difference to the way we used in the first place.

The solution is to decouple the process from the shell that system() opens. You do that by appending ‘&’ to the command you execute. And since heredocs are weird vertical strings you don’t append it in the bottom line after the second EOF but just like this:

1
2
3

script/unner ProcessRunner.run <<EOF &
...
EOF

The final version in the class looks like this:

system(“#{RAILS_ROOT}/script/runner  ProcessRunner.run <<EOF &\n#{YAML.dump(message)}\nEOF”)

And this is actually called by another method inside ProcessRunner:

1
2
3

def self.prepare(message)
  system(“...”)
end

By passing all responsibility for the preparation to the class that will eventually do it we can later on just switch so some other method, should this not fit our needs.

Popularity: 1% [?]