HiveBrain v1.2.0
Get Started
← Back to all entries
patternrubyrailsMinor

Reading a file in chunks

Submitted by: @import:stackexchange-codereview··
0
Viewed 0 times
readingfilechunks

Problem

I currently have the following class that takes a custom file object as an argument and makes a copy of the file locally.

class FileCopier
 def initialize(file_info_object)
  @file_info_object = file_info_object
 end

 def copy
  File.open(local_file_path, 'wb') do |f|
   read_file_in_chunks do |chunk|
     f.write chunk
   end
  end
 end

private

 def local_file_path
  File.join(Rails.root,'tmp/blah.txt')
 end

 def read_file_in_chunks(chunksize=10343)
  index = 0
  size = @file_info_object.size
  path = @file_info_object.path

  while index < filesize
   yield Client.read(path, offset: index, length: chunksize)
   index += chunksize
  end

 end

end


The copy method seems to be fairly straight forward but I was wondering if there is a way to simplify it. It seems difficult to test to me at the moment.

Solution

Some notes:

  • Use 2-space indentation.



  • f.write chunk. You use parens in other places, be consistent.



Other than these minor issues, I think your code is pretty good. I'd just refactor read_file_in_chunks to use a enumerator instead (more versatile) and a range.

def read_file_in_chunks(chunksize=10343)
  Enumerator.new do |yielder|
    (0...@file_info_object.size).step(chunksize).each do |offset|
      data = Client.read(@file_info_object.path, offset: offset, length: chunksize)
      yielder.yield(data)
    end
  end
end


Using lazy collections (Ruby >= 2.0) it can be simplified even more:

def read_file_in_chunks(chunksize=10343)
  (0...@file_info_object.size).step(chunksize).lazy.map do |offset|
    Client.read(@file_info_object.path, offset: offset, length: chunksize)
  end
end

Code Snippets

def read_file_in_chunks(chunksize=10343)
  Enumerator.new do |yielder|
    (0...@file_info_object.size).step(chunksize).each do |offset|
      data = Client.read(@file_info_object.path, offset: offset, length: chunksize)
      yielder.yield(data)
    end
  end
end
def read_file_in_chunks(chunksize=10343)
  (0...@file_info_object.size).step(chunksize).lazy.map do |offset|
    Client.read(@file_info_object.path, offset: offset, length: chunksize)
  end
end

Context

StackExchange Code Review Q#77158, answer score: 2

Revisions (0)

No revisions yet.