principlerubyMinor
Logging to a File vs UDP
Viewed 0 times
loggingfileudp
Problem
I'm trying to understand where my ruby implementation of this fails and hoping for an expert to shed some light on this situation. I've tried to make this a very simple example with the idea that setup is minimal.
The Problem:
I need to be able to log very quickly, ideally over UDP. Alternatively I can log to a file but I'd prefer not to for a variety of reasons but mainly logfile rotations, another dependency like a "forwarding" app, etc.
With the code below I am able to benchmark 100_000 [<= 16byte] log messages:
Specifics on that code include: ruby 2.0.0-p353 MRI
The Logfile Setup
UDP
The server (run this first):
The client:
Why UDP? Well I was under the impression that UDP was a "fire and forget" protocol where the client would not wait for the server to say they received it. Fire and forget is fine for me in this scenario, I care more about performance.
Issue 1
As outlined in "The Problem" area - why is UDP so much slower than writing to a file?
Issue 2
You'll notice I have a commented out
The Problem:
I need to be able to log very quickly, ideally over UDP. Alternatively I can log to a file but I'd prefer not to for a variety of reasons but mainly logfile rotations, another dependency like a "forwarding" app, etc.
With the code below I am able to benchmark 100_000 [<= 16byte] log messages:
Logfile UDP
Time elapsed (in seconds) 1.36 24.48Specifics on that code include: ruby 2.0.0-p353 MRI
The Logfile Setup
require 'logger'
logger = Logger.new("logfile.log")
start_time = Time.now
100_000.times do |i|
logger.info "#{i}"
end
puts "Elapsed time was #{Time.now - start_time}"UDP
The server (run this first):
require 'socket'
s = UDPSocket.new
s.bind(nil, 1234)
loop do
text, sender = s.recvfrom_nonblock(16)
# puts text
endThe client:
require 'socket'
s = UDPSocket.new
before_time = Time.now
100_000.times do |i|
s.send("#{i}", 0, 'localhost', 1234)
end
puts "Time elapsed was #{Time.now - before_time}"Why UDP? Well I was under the impression that UDP was a "fire and forget" protocol where the client would not wait for the server to say they received it. Fire and forget is fine for me in this scenario, I care more about performance.
Issue 1
As outlined in "The Problem" area - why is UDP so much slower than writing to a file?
Issue 2
You'll notice I have a commented out
puts statement in the server. If you uncomment that, the client's benchmark approaches 40+ seconds which really makes me think the client is waiting for the server or the server is communicating with the client. Either way I'm probably not understanding UDP correctly.Solution
Well I learned a lot today, specifically that writing to UDP is on average faster than writing to a file on my local machine and on my servers. My implementation on the UDP client was off.
If you look at this code:
There's something sneaky in there that I didn't catch with this line:
When you pass in host and port into the send method it DNS resolves before each send, which has major overhead. If instead I change the code to
I get benchmarks that beat File IO. Boom.
If you look at this code:
require 'socket'
s = UDPSocket.new
before_time = Time.now
100_000.times do |i|
s.send("#{i}", 0, 'localhost', 1234)
end
puts "Time elapsed was #{Time.now - before_time}"There's something sneaky in there that I didn't catch with this line:
s.send("#{i}", 0, 'localhost', 1234)When you pass in host and port into the send method it DNS resolves before each send, which has major overhead. If instead I change the code to
connect ahead of time like so:require 'socket'
s = UDPSocket.new
before_time = Time.now
s.connect('127.0.0.1', 1234)
100_000.times do |i|
s.send("#{i}", 0)
end
puts "Time elapsed was #{Time.now - before_time}"I get benchmarks that beat File IO. Boom.
Code Snippets
require 'socket'
s = UDPSocket.new
before_time = Time.now
100_000.times do |i|
s.send("#{i}", 0, 'localhost', 1234)
end
puts "Time elapsed was #{Time.now - before_time}"s.send("#{i}", 0, 'localhost', 1234)require 'socket'
s = UDPSocket.new
before_time = Time.now
s.connect('127.0.0.1', 1234)
100_000.times do |i|
s.send("#{i}", 0)
end
puts "Time elapsed was #{Time.now - before_time}"Context
StackExchange Code Review Q#86983, answer score: 7
Revisions (0)
No revisions yet.