in web we trust Fernando Blat's blog, freelance web developer

I'm going to start a new project which can have in very short time hard requirements on database performance: there are two entities, let's say items and users that will be related (a common one-to-many relationship). The question is that this relation is very probable to grow a lot and the item will be related with thousands of users.

In order to avoid having the one-to-many table with millions of rows I have decided to try with some NoSQL solutions, with the good luck that the first that I have tried still keeps me very impressed: Redis is very awesome! It's very fast, extremely easy to use and very powerful. And the set, ordered set or list seem to fit perfectly in my problem.

So I decided to perform some tests:

 total_actions = 100000
 # 10000 actions
 1.upto(total_actions) do |i|
   b = Benchmark.measure do
     subscribers_per_action = 10000
     1.upto(subscribers_per_action) do |user_id|
       $redis.incr "action-#{i}-count"
       $redis.sadd "action-#{i}", user_id
     end
   end
   puts "#{i} / #{total_actions} / #{b.real} / #{$redis.dbsize}"
 end
 

The results are impressive: 10,000 incr and 10,000 adds to a set performed in 2.4 secs approximately. And not only in an empty database. I'm testing it with databases that have a lot of entries and still is performing such well.

So I decided to try the same in a Tokyo Tyrant server. The problem is that Tokyo Cabinet doesn't support datasets, it is a classical key-value storage. So if you want to add elements in a key you have to store them in a YAML, or in a string with a well-known separator, or that way.

The same test with Redis took the double of time with Tokyo Tyrant. I know that 5.7 secs for 20,000 operations is very very fast, but I'm still impressed of the performance of Redis working with data types much more complex that a string. My choice is quite clear.

This weekend we have been rumbling, and as a result we present Tagueame. In a few words:

Tagueame is a little experiment about socializing online opinions about your friends and family.

Feel free to register and enjoy. It's very funny!

It is very ease make link_to_remote accessible, it is, forcing it has a href value different than #, which is the default value if you only indicate the :url parameter.

But, it is quite possible that both parameters have the same value. If so, you can redefine link_to_remote in this way:

 module ActionView
   module Helpers
     module PrototypeHelper
       def link_to_remote(name, options = {}, html_options = {}) 
         url = if options[:url].is_a?(Hash)
           url_for(options[:url])
         else
           options[:url]
         end
         html_options[:href] = url unless html_options[:href] 
         link_to_function(name, remote_function(options), html_options)
       end
     end
   end
 end
 

Related to my last post about testing cache and some curiosity about Shoulda I made a branch of it to add some helper methods for testing cache generation and expiration:

• should_cache_fragment
• should_cache_action
• should_expire_fragment
• should_not_expire_action
• ...

The good thing was to investigate how shoulda was organized, the test suite it has and that is so easy to hack.

I submited a ticket that I hope the authors approve in next days.

The last month I have been working in Query Memcached, a plugin that replaces the Query Cache that comes with Rails, adding a Memcache layer for persisting the query's cache between requests. That means that the cache does not expire at the end of the request, because it is stored in Memcached.

If you are not familiar with Query Cache, you only have to know that it's a very simple caching system from ActiveRecord that stores in memory all the queries performed during a request. If a query is fired more than one time, it will be in cache, so the database won't be affected.

The adventages of my approach are evident: you only use the database when some table has been modified or when a new query is executed. So you can get a notice an important speed improvement.

It is ideal for pages which have a lot of customization for the user logged in the application: in that cases it is very difficult to cache fragments or pages, because of the expiring of all that cache, and so many things.

Cache expiring

For expiring this cache, each Memcached key contains a sum of all the version numbers of the tables involved in the query. If one of that tables is modified, then the version number for that table is increased.

For example, the query below involves the table items and the table places:

So, the version for the cache of that query will be the sum of the cache version of the table items and the cache version of the table places.

Disclaimer

We are using this plugin in two on-line web projects: unvlog.com and iwannagothere, and everything seems to work fine (and faster than before), but these sites don't have so much traffic (about 30.000 req/day), so, there are stille possibilities that appear bugs and "unexpected behaviours".

In spite of all that, if you still are encouraged to try feel free to ask me anythin.

More information at README.

At least this week I had some time to spent in ending a litte small project: adapt and improve the cache_test plugin to Rails 2.x. You can find it at GitHub: cache_test.

Cache test is a very useful plugin that allows you to test if action/fragment/page cache is generated or expired during a request. For me is very surprisingly that Rails hasn't asserts for testing the cache, specially when, if your application uses cache, it can be the cause of its malfunctioning. Apart of that cache is so tedious to debug.

The way it works is very easy:

     assert_cache_fragment({:fragment => 'total_posts'}) do
       get :index, :user_id => users(:first)
       assert_response :success
     end
 

In my plugin I have adapted the cache storage definition to Rails 2.x (moved to ActiveSupport, etc), added more asserts and some tests.

Enjoy it!

More info in the README.

These days I have been working in a small parser to convert BBCode to HTML in Ruby.

The motivation was that I need it (of course), and there wasn't anything complete enough. So I found bb-ruby at GitHub and I decided to complete it with more tags, tests, and some stuff for doing it more flexible.

The result is a fork that you can find here:

http://github.com/ferblape/bb-ruby/tree/master

You can read the README file for getting more information.

A small snippet for testing.

If you want to add some special actions before and after the execution of each test, and don't want to add it to setup and teardown because you will be repeating your code for each unit test and functional test, or because it is a block, you can do an alias_method_chain for the run method:

 module Test
   module Unit
     class TestCase
       
 
       def run_with_my_block(*args, &block)
         Cache.clean # code before 
        
         a_block_that_you_want_to_execute do 
           run_without_my_block(*args, &block)
         end
         
         # Code after
       end
 
       alias_method_chain :run, :my_block
 
     end
   end
 end
 

You can redefine it at the beginning of your test_helper.rb for example.

Note that this code is only for Rails, becaus of the use of alias_method_chain function.