The only information I had was the generic error (an object was expected), and the line and column where the mistake was. So I decide to understand how to find that line in all the linked javascript files.

My conclusions are that the line could be from the HTML resulted from loading the page, or that line in one of each javascript files. It is, you have to find that line in all your javascripts and in the result page (viewing the source code) and then try to deduce which of all causes the error.

If the number of line is big enough you could reject some files, but not always you'll be so lucky.

And that's what I have learnt today.

The idea is that, given a request, the webserver has to decide if it is an static or a dynamic url and then serve the request itself or pass it to the proxy of application servers..

The problem is that, as thin and mongrel are also web servers, if you don't take care with the configuration your application server can be serving all the static files too and be busy when it has to attend a dynamic request, with all the problems it causes.

That's why for me is very surprising when you search in Google about Apache + Mongrel, for example, and you arrive to virtual hosts configurations that doesn't take this into account.

The rules are very easy:

 RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f
 RewriteCond %{REQUEST_URI} !/stylesheets/.* [NC]
 RewriteCond %{REQUEST_URI} !/images/.* [NC]
 RewriteCond %{REQUEST_URI} !/javascripts/.* [NC]
 RewriteCond %{REQUEST_URI} !.*\.js$ [NC]
 RewriteCond %{REQUEST_URI} !.*\.jpg$ [NC]
 RewriteCond %{REQUEST_URI} !.*\.gif$ [NC]
 RewriteCond %{REQUEST_URI} !.*\.css$ [NC]
 RewriteCond %{REQUEST_URI} !.*\.js$ [NC]
 RewriteCond %{REQUEST_URI} !.*\.png$ [NC]
 RewriteCond %{REQUEST_URI} !.*\.ico$ [NC]
 RewriteCond %{REQUEST_URI} !.*\.bmp$ [NC]
 RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L]
 

And this is an example: in order to your needs you can tune this configuration including more filet ypes and static directories.

For nginx the rule is also very simple.

So, you know, don't forge to include rewrite rules for serving static files.

This weekend we have been rumbling, and as a result we present Tagueame. In a few words:

Tagueame is a little experiment about socializing online opinions about your friends and family.

Feel free to register and enjoy. It's very funny!

But, it is quite possible that both parameters have the same value. If so, you can redefine link_to_remote in this way:

 module ActionView
   module Helpers
     module PrototypeHelper
       def link_to_remote(name, options = {}, html_options = {}) 
         html_options[:href] = options[:url] unless html_options[:href] 
         link_to_function(name, remote_function(options), html_options)
       end
     end
   end
 end
 

• should_cache_fragment
• should_cache_action
• should_expire_fragment
• should_not_expire_action
• ...

The good thing was to investigate how shoulda was organized, the test suite it has and that is so easy to hack.

I submited a ticket that I hope the authors approve in next days.

If you are not familiar with Query Cache, you only have to know that it's a very simple caching system from ActiveRecord that stores in memory all the queries performed during a request. If a query is fired more than one time, it will be in cache, so the database won't be affected.

The adventages of my approach are evident: you only use the database when some table has been modified or when a new query is executed. So you can get a notice an important speed improvement.

It is ideal for pages which have a lot of customization for the user logged in the application: in that cases it is very difficult to cache fragments or pages, because of the expiring of all that cache, and so many things.

Cache expiring

For expiring this cache, each Memcached key contains a sum of all the version numbers of the tables involved in the query. If one of that tables is modified, then the version number for that table is increased.

For example, the query below involves the table items and the table places:

So, the version for the cache of that query will be the sum of the cache version of the table items and the cache version of the table places.

Disclaimer

We are using this plugin in two on-line web projects: unvlog.com and iwannagothere, and everything seems to work fine (and faster than before), but these sites don't have so much traffic (about 30.000 req/day), so, there are stille possibilities that appear bugs and "unexpected behaviours".

In spite of all that, if you still are encouraged to try feel free to ask me anythin.

More information at README.

Cache test is a very useful plugin that allows you to test if action/fragment/page cache is generated or expired during a request. For me is very surprisingly that Rails hasn't asserts for testing the cache, specially when, if your application uses cache, it can be the cause of its malfunctioning. Apart of that cache is so tedious to debug.

The way it works is very easy:

     assert_cache_fragment({:fragment => 'total_posts'}) do
       get :index, :user_id => users(:first)
       assert_response :success
     end
 

In my plugin I have adapted the cache storage definition to Rails 2.x (moved to ActiveSupport, etc), added more asserts and some tests.

Enjoy it!

More info in the README.

The motivation was that I need it (of course), and there wasn't anything complete enough. So I found bb-ruby at GitHub and I decided to complete it with more tags, tests, and some stuff for doing it more flexible.

The result is a fork that you can find here:

http://github.com/ferblape/bb-ruby/tree/master

You can read the README file for getting more information.

If you want to add some special actions before and after the execution of each test, and don't want to add it to setup and teardown because you will be repeating your code for each unit test and functional test, or because it is a block, you can do an alias_method_chain for the run method:

 module Test
   module Unit
     class TestCase
       
 
       def run_with_my_block(*args, &block)
         Cache.clean # code before 
        
         a_block_that_you_want_to_execute do 
           run_without_my_block(*args, &block)
         end
         
         # Code after
       end
 
       alias_method_chain :run, :my_block
 
     end
   end
 end
 

You can redefine it at the beginning of your test_helper.rb for example.

Note that this code is only for Rails, becaus of the use of alias_method_chain function.

The problem is that it loads quite Javascript that has to connect to the main Google Maps page in order to draw our map. In order to avoid that we can use static Google Maps, it is, a picture (generated in real time) of the map we want to visualize. The problem with this is that you can't interact with the map (because it is an image) and that there is a limitation per day. But may be you find it useful in a particular moment.

For example, in iwannagothere.net we have a small map in every place that shows the location of that city:

In that situation we can change the map because the map is not for navigating along the city. For that you have a map in each item.

In Ruby on Rails, the programming framework of iwannagothere.net there is a gem called static-gmaps that does the hard work for you. What's more, we have found the way to integrate this maps with attachment_fu, in order you save a copy of the map in your filesystem and then avoid the day limitation usage.

Basically we have an attachment_fu kind of model (nothing special here):

class PlaceMap < ActiveRecord::Base
   belongs_to :place
 
   has_attachment  :content_type => :image, 
                   :storage => :file_system,
                   :processor=> :rmagick,
                   :path_prefix => 'public/userfiles/place_maps', 
                   :max_size => 1.megabyte
 
   validates_as_attachment  
 end
 

The we generate the static map instance:

	
 map = StaticGmaps::Map.new :center   => [ place.lat, place.long ],
                             :zoom     => 12,
                             :size     => [ 334, 144 ],
                             :key      => APP['gmkey']  
 

And save it in our PlaceMap model:

 PlaceMap.create(:uploaded_data => (open(map.url)), :place_id => place.id, :content_type => 'image/gif')
 

The problem here is that we are not uploading a file in the usual way: we are trying to save a raw of data. Attachment_fu doesn't know how to do that, so we have to apply a little patch:

 module Technoweenie
   module AttachmentFu
     module InstanceMethods
 
       def uploaded_data=(file_data)
         return nil if file_data.nil? || file_data.size == 0
         self.content_type = file_data.content_type
         self.filename     = Time.now.to_i.to_s+'.gif'
         if file_data.is_a?(StringIO)
           file_data.rewind
           self.temp_data = file_data.read
         else
           self.temp_path = file_data
         end
       end          
 
     end
   end
 end
 

Here we are giving a random filename, generated from the current time.

And that's all, really easy :)

iwannagothere.net is the last project from mamuso, Maria and me. It's a site about traveling with two main purposes:

1. share all the special places you have discovered in your travels: a singular monument, a delicious and cheap restaurant, a small museum with special charm, a park where you can see the most wonderful sunset ever...
2. help you to prepare your travels with all the information uploaded in the site

It's a kind of independent and high quality information travel guide.

As in unvlog.com we developed it in Ruby on Rails, (now running version 2.1), in our free time, along 3 months. Now we are refining a lot of details, and adapting some parts that we have seen people don't use as we thought.

We are going to explain all the technical issues we had in a serie of small posts in this blog or in the iwannagothere's blog. As well, we are going to present it at Spanish Rails Conference 08 in November.

We hope you enjoy the site and find it useful.

Un claro ejemplo lo encontramos cuando tenemos una relación :has_many y queremos comprobar si un elemento está incluída en dicha relación.

Por ejemplo:

 class Place < ActiveRecord::Base
   has_many :items
 end
 

 class Item < ActiveRecord::Base
   belongs_to :place
 end
 

Seguro que más de una vez, para comprobar que un place contiene un ítem hemos utilizado el "rubysta" include?:

 place.items.include?(item)
 

Pensándolo fríamente 0.2 segundos nos damos cuenta de que estamos cargando un array con todos los elementos de una relación, que pueden ser 10 o pueden ser 1.000.000. Y ya sabemos lo generoso que es Ruby con la memoria que toma y que nunca deja ir.

Un truco para evitar que esto suceda sin dejar de utilizar include? es definir dicho método dentro de la relación tal que así:

 class Place < ActiveRecord::Base
   has_many :items do 
     def include?(item)
       count(:conditions => ["item_id = ?", item.id]) > 0
     end
   end
 end
 

Es decir, resolvemos el problema con un simple COUNT de Mysql.

¿Alguien se anima a hacer un plugin? Creo que sería bastante inmediato para cuando se cumplen las convenciones y sabemos que la clave foránea de la relación es singular del nombre de la relación_id. O quizá ni eso, si encontramos la forma de preguntarle a ActiveRecord cuál es la clave foránea de una relación.

Sin embargo muchas veces con el log de Rails no es suficiente, y hay que alcanzar un mayor nivel de detalle.

El log en modo debug nos da todo el detalle que necesitamos, pero mostrando demasiada información (¿de qué nos sirve saber que un SELECT sencillo tarda 0.001 segundos?).

Pero si en ese log pudiéramos filtrar las consultas SQL más lentas sí que tendríamos información de consultas lentas asociadas a cada petición al servidor.

Gracias al plugin query trace y a una pequeña modificación podemos mostrar qué consultas tardan más de 1 segundo:

 module ActiveRecord
   module ConnectionAdapters
     class AbstractAdapter
       def log_info(sql, name, runtime)
         return unless @logger
         @logger.info(
           format_log_entry(
             "#{name.nil? ? "SQL" : name} (#{sprintf("%f", runtime)})",
             sql.gsub(/ +/, " ")
           )
         ) if (RAILS_ENV == 'production' && runtime > 1.0) || RAILS_ENV != 'production'
       end
     end
   end
 end
 

Este código lo he probado en Rails 1.2.x, no sé si funcionará en otras versiones.

Sin embargo yo utilizo un script que creo que me pasó porras para que me muestre en colores las modificaciones.

El script es este:

 #!/usr/bin/env ruby
 
 `svn diff #{ARGV.join(' ')}`.each do |line|
   puts( if line =~ /^\+(.*)$/
         "\e[32m#{$&}\e[0m"
         elsif line =~ /^-(.*)$/
           "\e[31m#{$&}\e[0m"
         else
           line
         end
       )
 end
 

Con darle permisos de ejecución ya podéis tener un svndiff por ejemplo.

No pongo el equivalente en git porque es bastante inmediato.

• claves: máximo de 250 caracteres
• valores alojados: máximo 1 Megabyte

Es muy difícil que una aplicación Rails las claves generadas a partir de las URL's tengan tanto tamaño, pero puede darse el caso y hay que tenerlo presente.

En el grupo de discusión acts_as_cached esta semana han propuesto hashear (con MD5, por ejemplo) las claves antes de guardarlas para curarnos en salud.

El problema es el siguiente: en unvlog.com (sí, ese proyecto del que no he hablado) somos muy Rails y tenemos para todos los recursos que tiene sentido una ruta que responde a un formato u otro según se le solicite pero con esta forma:

En concreto, esta URL corresponde al feed de un blog, feed que está cacheado en disco gracias a la caché de página de Rails. Pues bien, la idea es que el servidor web sirva directamente esta caché, así que una regla podría ser:

   if (-f /cache/$request_filename) {
           rewrite (.*) /cache/$1 break;
   }
 

(Esto, asumiendo que tenemos la caché en public/cache)

El problema es que esta regla no funciona, y creo que se debe a que blat.atom lleva un '.' y no es capaz de asignar la URL a la variable $request_filename.

Al final ha sido tan fácil como utilizar otra variable que he encontrado en la parte rusa del wiki, $request_uri:

   if (-f $document_root/cache/$request_uri) {
           rewrite ^/(.*).atom$ /cache/$1.atom break;
   }
 

Con esto ha funcionado perfectamente y ya tenemos cacheados y servidos por Nginx los feeds en unvlog.com.

Para acceder a él a través del cliente de Ruby, por ejemplo, basta con cargar una consola en el entorno deseado e invocar al método stats del objeto de caché.

Por ejemplo, si nuestra configuración es la siguiente:

   CACHE = MemCache.new 'localhost:11211', :namespace => 'wadus_fragments'
 

Podemos utilizar el objeto CACHE de la siguiente manera:

   pp CACHE.stats
   {"bytes"=>746532339,
    "pid"=>10303,
    "connection_structures"=>19,
    "time"=>1200349556,
    "limit_maxbytes"=>1073741824,
    "cmd_get"=>689486,
    "version"=>"1.1.12",
    "bytes_written"=>3296488315,
    "cmd_set"=>265986,
    "get_misses"=>265168,
    "total_connections"=>1238,
    "curr_connections"=>17,
    "curr_items"=>66977,
    "uptime"=>56490,
    "get_hits"=>424318,
    "total_items"=>265986,
    "rusage_system"=>95.012555,
    "rusage_user"=>17.16339,
    "bytes_read"=>3226552602}
 

La interpretación de todos estos valores se puede encontrar en la definición del protocolo:

   Name              Type     Meaning
   ----------------------------------
   pid               32u      Process id of this server process
   uptime            32u      Number of seconds this server has been running
   time              32u      current UNIX time according to the server
   version           string   Version string of this server
   pointer_size      32       Default size of pointers on the host OS
                              (generally 32 or 64)
   rusage_user       32u:32u  Accumulated user time for this process 
                              (seconds:microseconds)
   rusage_system     32u:32u  Accumulated system time for this process 
                              (seconds:microseconds)
   curr_items        32u      Current number of items stored by the server
   total_items       32u      Total number of items stored by this server 
                              ever since it started
   bytes             64u      Current number of bytes used by this server 
                              to store items
   curr_connections  32u      Number of open connections
   total_connections 32u      Total number of connections opened since 
                              the server started running
   connection_structures 32u  Number of connection structures allocated 
                              by the server
   cmd_get           64u      Cumulative number of retrieval requests
   cmd_set           64u      Cumulative number of storage requests
   get_hits          64u      Number of keys that have been requested and 
                              found present
   get_misses        64u      Number of items that have been requested 
                              and not found
   evictions         64u      Number of valid items removed from cache                                                                           
                              to free memory for new items                                                                                       
   bytes_read        64u      Total number of bytes read by this server 
                              from network
   bytes_written     64u      Total number of bytes sent by this server to 
                              network
   limit_maxbytes    32u      Number of bytes this server is allowed to
                              use for storage. 
   threads           32u      Number of worker threads requested.
                              (see doc/threads.txt)
 

En concreto cuatro parámetros me han parecido interesantes de observar:

• el parámetro bytes junto con el de limit_maxbytes: el primero representa el total de bytes ocupados actualmente, frente el máximo.
• el número de fallos de caché que refleja el valor del parámetro get_misses junto con el número de aciertos get_hits

Los primeros nos permiten saber si hemos llegado al límite el almacenamiento que necesita nuestra aplicación y el segundo nos permite saber si nuestra política de caché es adecuada: diviendo hits entre misses obtenemos la proporción de fragmentos encontrados frente a fragmentos que ya no existían. Un valor de 1 indica que por cada dos búsquedas, una tiene éxito y otra no.

Por supuesto un valor inferior a uno muestra que la caché no sirve de nada (indicaría por ejemplo que necesitaríamos ampliar el Memcache o que tenemos una política de borrado "exagerada").

Caché en Rails

Pronto, todas las demás, colgadas en la web de la Conferencia,

Además, en el post utilizan Nginx como servidor web y Ruby on Rails y Merb, como frameworks de desarrollo, así como Memcached para almacenar sesiones.

Los resultados, muy interesantes:

• Merb es mucho más rápido que Rails, a costa de perder parte de la magia
• Nginx es una gran alternativa a Apache, sobretodo cuando la memoria es un recurso escaso (en mi nuevo Slicehost, por ejemplo)
• y, el más importante, el evented Mongrel funciona bastante mejor que el Mongrel en situaciones de alta concurrencia

Así que si te gusta "jugar" y quieres probarlo, lo recomiendo: la instalación es muy sencilla y los resultados saltan a la vista.