imedo Development Blog

If you need a quick overview whether the internal links of your website are working and the markup is valid, checkout the Multipage Validator tool.

Popularity: 1% [?]

Nice approach for circumventing input validation for JSON: Parameter Pollution with JSON.

Be sure to have your input validation handle this kind of injection attacks.

Popularity: 1% [?]

We’re pretty much a Mac shop here at imedo and, as our application is built using Ruby on Rails, it’s safe to say that we are legally obliged to use TextMate for all our editing needs .

One of the features I love about TextMate, catered for in most bundles, is the ability to execute a script and get a nice pop-up window displaying the results. Scrolling through my installed language bundles, I see they all have a “Run Script” command. All, that is, except my second favourite language – Javascript. You fancy rectifying this? You fancy giving Javascript some TextMate/V8 lurv? Then walk this way …

First, lets grab V8. We’ll be using scons to build it. Scons? Aren’t they a strange  muffin-like concoction originating in the fair British Isles? That’ll be ‘scones‘ you’re thinking of. Scons is a software construction tool written in Python. Check it out at here. It must be good – Zed likes it. Install with macports.

And on with the build.

Get v8:

$> svn co http://v8.googlecode.com/svn/trunk v8
- or -
$> git clone git://github.com/v8/v8.git v8
…
$> cd v8
$> scons

The build should have been successful. V8 comes with a shell which we’re going to use. Build like so:

$> g++ ./samples/shell.cc -o v8-shell -I include libv8.a

Make sure that ‘v8-shell’ is in your PATH. And now for the final step. In TextMate: Bundles > Bundle Editor > Show Bundle Editor. Scroll down to the Javascript bundle and Create a new Command. We’ll use our imagination here and call it “Run Script”. In the Edit Command Window:

Save: Nothing
Command(s):


$(type -p "${TM_RUBY:-ruby}") -e'
require ENV["TM_SUPPORT_PATH"] + "/lib/tm/executor"
require ENV["TM_SUPPORT_PATH"] + "/lib/tm/save_current_document"
TextMate.save_current_document
TextMate::Executor.make_project_master_current_document
TextMate::Executor.run("v8-shell", ENV["TM_FILEPATH"])'


Input: Entire Document
Output: Show as HTML
Activation: Key Equivalent > Cmd+R
Scope Selector: source.js

This is copied and adapted from the “Run Script” Command for the Ruby bundle.

And that – as they say – is that. Time to check it out:

Open a new window in TextMate and type the following:

function addSomeValues (first, second) {
return first + second;
}

print(addSomeValues(2,3))

Select the Javascript scope. Wait for it … Ready? … Go Cmd+R! Hopefully you saw the number five.

Now change the last line to read

print(addSomeValueCATsaTOnKeyboArD(2,3))

Run again and you should get a nice ReferenceError.

Thanks go to gs over at StackOverflow for this post detailing the steps for getting the v8 shell built.

Popularity: 1% [?]

Another short tip, because I really like the smusher gem!

first install it:
gem install smusher

then crush your images automatically using the PunyPng service, which is currently delivering the best results in terms of file size:
smusher * --service PunyPng

Popularity: 1% [?]

If you need to generate thumbnails of the first pages of alot of PDF documents you can use this tiny ruby script:
http://bit.ly/8T0B1a
I also included smusher support to crush the PNGs after they were created. Just gem install smusher and uncomment the smusher command.

Enjoy!

Popularity: 1% [?]

Although discovered already in 1988 by Norm Hardy, cross-site request forgery (CSRF) has been the shooting star of web attacks in 2008. As a result it has become one of the 2009 CWE/SANS Top 25 Most Dangerous Programming Errors.

The idea behind CSRF is that an attacker sends a malicious request to the target application using a trusted connection which he expects to have been established by an innocent user. In case of web applications this could be done by hiding the request in a web page with harmless content. If the user visits the malicious web page while he is logged in to the target application in another browser tab, the dangerous request is send to the target over the trusted connection between browser and web application. Two basic scenarios are possible. In the first scenario the attacker forges a request to a commonly used web application so that it doesn’t matter who opens the malicious web page. The other scenario is to trick somebody who is known to have special privileges for one web application to open the prepared web page.

In both scenarios the malicious web page of the attacker is crafted so that it sends a hidden request to the target application. For example a HTTP GET request could be executed by using the src attribute of an img tag like in the following line of code. In this case it would delete one user account.

<img src="http://victim.example.com/user/destroy/1" />

Another possibility to hide a request in a web page is displayed in the next example. It’s a hidden HTTP POST request which changes the name and email address of the logged in user. It sends the request automatically on page load. With the changed email address the attacker can use the “forgot password?” function in order to receive a newly generated password and capture the account.

<SCRIPT>
  function SendAttack() {
    var form = document.createElement("form");
    form.style.display = "none";
    this.parentNode.appendChild(form);
    form.method = "POST";
    form.action = "http://victim.example.com/profile.php";
    form.first_name = "Bad";
    form.last_name = "Guy";
    form.email = "attacker@example.com";
    form.submit();
  }
</SCRIPT>
<BODY onload="javascript:SendAttack();"></BODY>

These two examples demonstrate that HTTP GET as well as HTTP POST are vulnerable by CSRF. Hence it is not enough to allow only POST requests to alter data. Other measure need to be applied, too. An obvious client-side countermeasures is to use separate browsers for surfing and administration, but Rails provides a server-side measure as well. The method protect_from_forgery() automatically inserts a security token in all forms and Ajax request generated by Rails. The token is calculated from the current session and the server-side secret. If the session storage is not CookieStorage the secret needs to be passed as option to the method. If an attacker is trying to send a request without the valid token to the application an ActionController::InvalidAuthenticityToken error is raised.

In Rails versions newer than 2.0 protect_from_forgery() is called for all HTTP POST requests by default. Hence it only has to be taken care of that HTTP GET  and HTTP POST requests are used appropriately (see W3C checklist) and that this is enforced. For assuring that actions are only called by HTTP POST requests Rails provides the verify() method which can be added to a controller as demonstrated in the following example. It is also shown how to define actions which should be callable by other HTTP methods than POST with the except option.

class MyController < ApplicationController
  verify :method => "post", :except => :index
end

In older Rails versions the protect_from_forgery() method can be used like shown in the following lines of code. In this example CSRF protection is disabled for the index action.

class MyController < ApplicationController
  protect_from_forgery :except => :index
end

For further information on cross-side request forgery see www.cgisecurity.com/csrf-faq.html.

Popularity: 1% [?]

The transmission of sensitive information in cleartext is the number six of the 2009 CWE/SANS Top 25 Most Dangerous Programming Errors. Sending data without encryption becomes a security issue when nobody but the receiver is supposed to be able to read the included information and the data is send over an open network like the Internet. Login information for closed areas on websites is a typical example for this kind of data.

How easy it is to sniff for sensitive information on the Internet has been demonstrated with the  Tor-pishing attack not long time ago. As a result it is highly recommended to encrypt the whole communication between login and logout on access controlled web pages.

One way to secure communication over the Internet is to activate the transport layer security (TLS), formerly known as secure socket layer (SSL), for web applications. TLS encrypts the transmitted data between web browser and web server which denies unauthorized access to the sensitive data. Recently a vulnerability in TLS has been discovered and exploited. But common implementations like OpenSSL have already been fixed.

With the help of the SSL requirement plugin (github fork of Ian Warshak) secure communication can be easily added to Rails applications. It can be used opt-in or opt-out. The plugin is added to an application with the following lines of code.

class ApplicationController < ActiveRecord::Base
  include SslRequirement
end

In order to implement encryption opt-in, the plugin should be used like in the following example. In this example the actions signup and login are set to require TLS which means that non-TLS requests are forwarded to TLS. The index action can be accessed with or without TLS.

class AccountController < ApplicationController
  ssl_required :signup, :login
  ssl_allowed :index
end

For applications which are supposed to have encryption enabled by default and only some actions should be accessable without TLS, the plugin provides opt-out for TLS as well. In the following example all actions but index require TLS. Calling ssl_exceptions() with no action enables TLS for all actions.

class AccountController < ApplicationController
  ssl_exceptions :index
end

The SSL requirement plugin provides TLS support in Ruby on Rails by checking whether the HTTP header X_FORWARDED_PROTO of the request is set to ‘https’. This is done with the method request.ssl? of Rails. The plugin does not implement TLS. Hence further configuration of the used web server is needed. How to configure a web server with TLS is beyond the focus of this article, but be sure to consider the case that an attacker might set X_FORWARDED_PROTO to ‘https’ in an ordinary HTTP request. Rails depends on the underlying web server to create and handle the actual TLS connection.

UPDATE: Further information about TLS is provided on OWASP’s TLS cheat sheet.

Popularity: 1% [?]

This post might be interesting for all people who are faced with strange problems like this: “Yesterday it worked. Now it’s broken” or “It works on my machine (and it does not in production)”.

I’m sure that all Programmers and sysadmins have had an incident like this in their lives. I’ve had a lot of these problems and found out that in the end there’s always an explaination for the problem. Very rarley it’s some quantum mechaincs effect that caused the problem. In most cases there is a really simple explaination for the problem even if it was hard to find. These things include “the cause of a crashing perl script is the java version of the app starting the script”, “failing tests that where caused by a minor version difference of a testing library where the error message lead to something completely different”

The process described below was used in all cases to find the root cause of the problem which then was solved very easily. We weren’t aware that we used this process but rather did it intuitively. After talking about the process and writing it down, we were able to find more and more problems by following these steps and also to transfer the knowledge to other people so that they will build up the intuition to find root causes of problems as well.

General idea

Systems that work and system that don’t work differ.

If you make the not working system equal to the working system, it will work.

That’s all there is to Troubleshooting (basically).

Process to find out the difference

The hard part is to find out where the working and not working systems differ.

The general process is really simple though:

1. List all items that can differ
2. Check if they differ
3. Make them equal (one at a time!)
4. Repeat 3 until finished. If still broken, think harder about 1 and start again

The optimized version of this is, to start with the things that are most likely to cause the problem.

The term “most likely” is based on

• your own experience
• information found in the web: blog posts, google searches, etc.
• experiences of your co-workers

It is fundamental that you make each step conciously (writing the step/change down helps to do that). If one step doesn’t yield the desired outcome: revert it immediately. Again: having written it done helps not to forget anything. Forgetting steps may make the situation even worse.

How can systems differ?

The main questions are:

• What changed since it worked (if it is the same system)?
• What is different or changed on the not working system compared to the working system (if it is a different system)?

The latter is a lot easier because you something to compare to. In the former case you have to create the “working system” again. Which in itself may be the solution to the problem.

If the answer is “nothing”. Think again…! Because time has progressed. So at least the time changed.

Possible effects of changed time

• File system full
• weird time dependend behavior of applications
• system/application restart occured
• data changes happend

Other things that may have changed:

• software versions through package updates – Minor Changes are important!
  • OS Kernel
  • OS packages
  • application libraries (ruby gems, jars))
• Database schemas
• Database content
• Filesystem content of any kind (That includes timestamps of a file that is only read!)
  • Location of files
  • symlink vs. real files
  • timestamps
• Hardware
• Increased load
  • Network I/O
  • Disk I/O
  • CPU
  • Exceeded RAM -> Swapping

Some things will be straightforward and it is obvious why something brakes something else. Some things are not as obvious (at least not at the time when you try to find it – it’s always obvious afterwards!). Don’t jump to conclusions about cause and effect while you debug. If you think “I’ll don’t try X because X has nothing to do with Y” try X! Maybe it has something to do with Y. You don’t know before you try. Revert (or create the equal state to the working system for) the “most obvious” things that “can’t possibly interfere with the problem”. That includes

• Comments in Source or Configuration files
• Whitespaces
• Trivial Code/Configuration changes
• minor version changes in Packages

The “most likley” rule does apply here, too. Don’t start with whitespace if there are other not so subtle changes still different. Don’t look for access time timestamps if the files on one system are are in completely different locations compared to on the other system. This requires some experience but with time you’ll find which things to look for first.

Tools

• Filesystem-Analysis: df, ls, find,
• Application-Behavior: strace (dtruss on Solaris and Mac OS), lsof, netstat
• Databases: For mysql: mysql, innotop
• Packages: Debian: apt-get, dpkg
• Finding differences/problem causes in running vs. not running code: Binary Search (e.g. via git bisect, debugger or just plain “print”-Debugging).

We hope these thoughts help you to debug and troubleshoot strange problems. Feel free to post additions, comments, tool or experiences with troubleshooting.

Popularity: 1% [?]

One of the biggest problems in software development has nothing to do with coding. It is about communication, the “code monkey versus software artist problem”.

Business people tend to specify as much as possible, but often lack the necessary knowledge of the technical details. From their point of view software developers are code monkeys who are supposed to implement specifications. On the other side programmers often see themselves as software artists who need total freedom since they are the ones who know the code.

The truth lies – like in most cases – somewhere in between. Feature requests are a result of business decisions which are made in the e.g. marketing or sales departments and usually need to be realized as soon as possible. But only software developers can estimate the needed resources to implement these features. They know which particular tasks have to be done, since they know the code.

One process which currently seems to be the solution for bringing business and software development together is agile software development with Scrum. It uses the rules of extreme programming (XP) and is therefore lightweight with low specification overhead and short release cycles. But still some specification needs to be done. Although the user stories of XP are really short they can contain quite a bit on information if they follow certain rules. Here are six rules which can help to get more information from one story. These rules do not conflict with the advices for writing stories in XP, but can be seen as a more practical approach which is based on them.

1 One use case per story

The first rule is to limit one story to one use case, i.e. only one use case for one feature is added or altered. For example if a new feature called “simple web forum” with the following use cases needs to be implemented.

• Guest views forum entry
• User writes forum entry
• User edits forum entry
• Moderator deletes forum entry

Each of these use case should be developed as one single story.

2 Consistent story format

Rule number two is to use a consistent format for the stories. The feature description part of the language Gherkin used by Cucumber is a good way of keeping stories short, but readable. In the following example is shown how it can be used to describe a use case.

Use Case: User writes forum entry
  In order to produce user-generated content for one topic
  As a logged in user
  I want to write a new forum entry

One use case description consists of four lines. The first line is a short summary of the use case. Line number two, the “in order to” line, is reserved for a brief business value explanation. And the last two lines are supposed to describe the uses case itself: Who needs to do what.

3 Cover all wanted scenarios with a story

Rule number three is about the used semantic. A story should always be formulated as generic as possible so that it covers all wanted scenarios. This is because giving only a specific scenario like in the following bad example means that only this scenario should be implemented.

Bad example:

Use Case: Little Red Riding Hood searches for "Wolf"
  In order to see how the Big Bad Wolf looks like
  As Little Red Riding Hood
  I want to search for "Wolf"

Good example:

Use Case: Guest searches for keyword
  In order to get information about a keyword
  As a guest
  I want to search for a keyword

4 Use of scenarios to describe use case

Sometimes the expected behavior for a scenario of a use case is different than for others, e.g. if access control is an issue. In this case the possible scenarios should be described in the story as well. Here, the scenario syntax of Gherkin can be used.

Use Case: User searches for content
  In order to get information about a keyword
  As a user
  I want to search for a keyword

  Scenario: User searches for private content
    Given a user Big Bad Wolf
    And a user named Little Red Riding Hood
    And Big Bad Wolf's private document with information about the Big Bad Wolf
    When Little Red Riding Hood searches for "Wolf"
    Then Little Red Riding Hood should not see the document about the Big Bad Wolf

5 Roles need to be known to the system

The fifth rule is that a role in a use case description of a story should always be known to the system, otherwise it is supposed to be added to the system. Like in the examples of rule 3 and 4 the Little Red Riding Hood might be a user name, but is probably no defined role. But e.g. the role “guest” stands for an unregistered visitor.

6 Use acceptance criteria for additional specification

In order to define additional things for one story it should be possible to give acceptance criteria for the QA team. This is rule number six. Acceptance criteria should be short and in list form. For the example with the keyword search one could be the following.

- A search form should be on every page

Popularity: 1% [?]

OS command injection is ranked number five on the 2009 CWE/SANS Top 25 Most Dangerous Programming Errors listing. It means to extend input, which is meant to be used as parameter for a shell command, with malicious shell commands. This is possible since operating systems like UNIX support the execution of several commands in one line by separating the commands with a ”;”.

The following line of code could be part of a simple web interface to Subversion. It downloads a copy of the repository given by the user by executing a system call to the svn-checkout command with the provided repository as parameter.

system "svn checkout #{params[:repository]}"

While this code is doing what it is supposed to, it also represents a great risk for the system’s security, i.e. its non-repudiation. Web applications which pass input values to system calls unchecked, enable an attacker to easily access the command line on the web server’s host with the privileges of the web server. E.g. if an attacker enters the string “https://svn.mydomain.org/code; rm -rf /var/www/*” for the above example he might delete all web pages on the host.

A countermeasure against OS command injection in Ruby on Rails is the usage of the system(command, parameter) method which ensures that no part of the parameter is handled as a shell command.

system("svn", "checkout #{params[:repository]}")

Popularity: 1% [?]