Posts Tagged ‘scalability’

Be cool with DRb, it's far from "scalable"

5 Oct
Posted by Nicolas in Ruby

When you begin to learn DRb, you quickly land on the famous Chad Fowler page, entitled “Intro to DRb”.

The “Concurrency” chapter is particularly interesting when you want to make a local resource available in the wild, allowing one request at a time on your resource.

So, with DRb, a dash of method_missing and a pinch of mutex, you have a perfect recipe to remotely access and protect your resource… but: DRb is not what we use to call a high availability entry point.

Let’s code the proof. Here is the code of the DRb server, it simulates a long and expensive task:

require 'drb'
 
class Server
 
  def initialize()
    @i = 0
    @mutex = Mutex.new
  end
 
  def method_missing(name, *args)
    @mutex.synchronize do
      @i += 1
      p @i
      sleep 1 # You CPU works very hard here..
      end
    @i
  end
 
end
 
server = DRb.start_service("druby://:34100", Server.new)
p "listening"
server.thread.join

You can now start the server via the ruby server.rb command and begin to code the client. It creates 100 processes, each one calling the DRb server and writing the response on the standard output:

require 'drb'
 
client = DRbObject.new(nil, "druby://:34100")
 
pids = []
 
100.times do
  pids << fork {
  p "#{Time.now}: #{client.call}"
}
end
 
p "#{Time.now}: created the 100 processes"
 
pids.each { |pid| Process.waitpid(pid) }
 
p "#{Time.now}: done"

Launch the client and see what happen:

"Sat Oct 05 11:54:17 +0200 2008: 2"
"Sat Oct 05 11:54:17 +0200 2008: 3"
"Sat Oct 05 11:54:20 +0200 2008: created the 100 processes"
"Sat Oct 05 11:54:17 +0200 2008: 4"
"Sat Oct 05 11:54:17 +0200 2008: 5"
"Sat Oct 05 11:54:17 +0200 2008: 6"
"Sat Oct 05 11:54:17 +0200 2008: 7"
"Sat Oct 05 11:54:17 +0200 2008: 8"
"Sat Oct 05 11:54:17 +0200 2008: 9"
"Sat Oct 05 11:54:18 +0200 2008: 10"
.......
"Sat Oct 05 11:54:18 +0200 2008: 59"
"Sat Oct 05 11:54:18 +0200 2008: 60"
"Sat Oct 05 11:54:19 +0200 2008: 61"
"Sat Oct 05 11:54:19 +0200 2008: 61"
DRb::DRbConnError: druby://:34100 - #
 
method open	in drb.rb at line 736
method each	in drb.rb at line 729
method open	in drb.rb at line 729
method initialize	in drb.rb at line 1189
method new	in drb.rb at line 1169
method open	in drb.rb at line 1169
method method_missing	in drb.rb at line 1085
method with_friend	in drb.rb at line 1103
method method_missing	in drb.rb at line 1084
at top level	in client.rb at line 9
method fork	in client.rb at line 8
at top level	in client.rb at line 8
method times	in client.rb at line 7
at top level	in client.rb at line 7

Almost the 100 processes are created before the DRb server returned its first calculation. After several tries on a PowerBook and an EC2 instance, the DRb server rejects any new client after about 65 simultaneous requests.

But, it’s important to mention that the DRb server did not crash at all. You simply have to wait that it handles the remaining requests.

DRb is pretty good, allowing the Ruby developers to code remote services in no time.

If you need high availability services, your next step could be REST servers, distributed/dispatched thanks to HAProxy or nginx. And of course, you should also take a look at Erlang.

Read more | Post your comment