in web we trust Fernando Blat's blog, freelance web developer

I'm going to start a new project which can have in very short time hard requirements on database performance: there are two entities, let's say items and users that will be related (a common one-to-many relationship). The question is that this relation is very probable to grow a lot and the item will be related with thousands of users.

In order to avoid having the one-to-many table with millions of rows I have decided to try with some NoSQL solutions, with the good luck that the first that I have tried still keeps me very impressed: Redis is very awesome! It's very fast, extremely easy to use and very powerful. And the set, ordered set or list seem to fit perfectly in my problem.

So I decided to perform some tests:

 total_actions = 100000
 # 10000 actions
 1.upto(total_actions) do |i|
   b = Benchmark.measure do
     subscribers_per_action = 10000
     1.upto(subscribers_per_action) do |user_id|
       $redis.incr "action-#{i}-count"
       $redis.sadd "action-#{i}", user_id
     end
   end
   puts "#{i} / #{total_actions} / #{b.real} / #{$redis.dbsize}"
 end
 

The results are impressive: 10,000 incr and 10,000 adds to a set performed in 2.4 secs approximately. And not only in an empty database. I'm testing it with databases that have a lot of entries and still is performing such well.

So I decided to try the same in a Tokyo Tyrant server. The problem is that Tokyo Cabinet doesn't support datasets, it is a classical key-value storage. So if you want to add elements in a key you have to store them in a YAML, or in a string with a well-known separator, or that way.

The same test with Redis took the double of time with Tokyo Tyrant. I know that 5.7 secs for 20,000 operations is very very fast, but I'm still impressed of the performance of Redis working with data types much more complex that a string. My choice is quite clear.

Tomorrow I'm leaving to Baltimore, I'll be attending to the RailsConf 2010 (as part as the Spanish noisy crew). It will be my first American Rails Conference (three years ago we went to London, when RailsConf was also european.

And in the next week I'll be in New York, picking our Webby award as part of the founder team of i wanna go there. So exiting!

The Spanish community Rails conference (aka Conferencia Rails) is so close! Last week we opened the register (what are you waiting for?), late than usual, but this year everything is a little bit more messy, because this year the conference is growing in a lot of aspects:

• this year we are moving to Vicálvaro, a village close to Madrid, to allow more attendees that past years (up to 250)
• we have 3 international keynoters: Yehuda Katz, David Black and Nathaniel Talbott
• also there are coming people from the States, like Scott Chacon from Github, or Obie Fernandez and the UK (surprise, surprise...)
• we are having an extra day of workshops

In spite of all the "anglo-attendees", the most of the talks will be in Spanish, and we are trying to make a calendar with talks in English and Spanish, not both in English at the same time.

There is a lot of diversity in the topics of this year: git, jruby, cucumber, sinatra, ruby 1.9, rails 3, internazionalization, key-value storages, ror-es mailing list, and so on... In fact, I'm so excited, and I think (I know which are the talks of this year) it is, maybe, one of the best editions of this conference.

Since we have been using I18n in Partigi for the last months we have notice that we were always repeating ourselves when localising a sentence like this:

This film has been saved by 4 friends

What is special in this sentece is that, depending on the number of friends that saved the film, the sentence could be "This film has been saved by one friend". It can be solved with pluralize helper, but it requires the counter to be at the beginning of the sentence.

Finally we decided to adopt a small convention: if a translation has a parameter count then, if count is singular the key for the translation will be the one in the view, but if it's plural, it will be the one in the view plus a suffix _plural. For example:

Before we had:

   <% if @reviews_count == 1 %>
     <%= t('films.saved_by_friends', :count => @reviews_count) %>
   <% else %>
     <%= t('films.saved_by_friends_plural', :count => @reviews_count) %>
   <% end %>
 

And now, with this hack:

   <%= t('films.saved_by_friends', :count => @reviews_count) %>
 

The hack is this (put it where you like more):

 module I18n 
   class << self
     # Returns true if a given key exists
     def exists?(key, options = {}) 
       locale = options.delete(:locale) || I18n.locale
       backend.exists?(locale, key, options = {})
     end
     
     # Overwrite I18n.trasnlate method
     def translate(key, options = {})
       locale = options.delete(:locale) || I18n.locale
       if options[:count] && options[:count].to_i != 1
         plural_key = "#{key}_plural" 
         if exists?(plural_key)
           key = plural_key
         end
       end
       backend.translate(locale, key, options)
     rescue I18n::ArgumentError => e
       raise e if options[:raise]
       send(@@exception_handler, e, locale, key, options)
     end
   end
 end
 
 

The good thing is that if the pluralized key doesn't exists, your views won't be broken, so you can start using it now and change your views when you have time.

One of my discoveries in the last Euruko conference was the project request log analyzer, an improved log analyzer for Rails logs (but also merb and the format you decide). Until now we have used production log analyzer from seattlerb boys, which is still a great tool.

The reason for this change is that request log analyzer let's you define your own parser which was exactly what we needed to parse logs from two of our applications (iwannagothere and partigi.com), because both of them have a mobile version in a different host (the common m.domain host) and we wanted to separate the analysis of these two versions as if they were different applications (they are different, really).

So modify the rails parser was really easy: we only had to change the rails completed line to match the domain of the request:

 # Completed in 614ms (View: 120, DB: 31) | 200 OK [http://floorplanner.local/demo]

In this example the domain is floorplanner.local. By default, the regular expression matches the full url: floorplanner.local/demo, but this url is not used in the analysis, so change it didn't supposed a drastic change.

So, we have copied the rails file and modified the rails completed line, with this new regular expression:

 RAILS_22_COMPLETED = /Completed in (\d+)ms \((?:View: (\d+), )?DB: (\d+)\) \| (\d\d\d).+\[http:\/\/([^\/]+)\/.+\]/

Now, you have to run request log analyzer indicating the path to your new parser and the parameter --select url m.your_domain. I.e.:

 /usr/bin/request-log-analyzer /var/www/partigi/shared/log/production.log --file /tmp/log_partigi --format /var/www/partigi/current/vendor/request-log-analyzer/rails_host.rb --select url m.partigi.com

Easy, isn't it?

It's time again for the Spanish Rails Conference! If you want to give a talk, you can send your proposals here. This year we want to encourage English people who wanna come with us and give talks (but notice that the most of the talks will be in Spanish).

Remember that this year there will be not only talks, but workshops of about 2 or 4 hours. The schedule is still not decided.

For more information, visit the oficial web.

Lately, I have been so curious about alternatives to relational databases. Deeping a little bit exist a lot of interest about topics related with it.

The most important that I have found recently is the NoSQL conference, with lot of talks and information about Voldemort, Cassandra, CouchDB, and so on.

You can find some notes about that here.

I'm still suspicous about all this technologies, specially because it implies a hard change in the way to model and consume data, moreover when SQL and RDMBs are very valid nowadays, and (for sure) they have a big future, but they offer a lot of advantages that you have to know in order to take advantage if you have the chance.

Update:

A great post about this topics: Should you go Beyond Relational Databases?

Since I saw the presentation about Couch Potato, a library for using CouchDB with your Ruby applications I was very curious about how to map a relational schema (that is the most common way to model entities and their relationship) in a document / plain schema where each element has no relation with the others.

And at last, today I found a cool post from Google and it's equivalent for CouchDB.

Now it's time to play with CouchDB!