HiveBrain v1.2.0
Get Started
← Back to all entries
snippetrustModerate

How can I find out why this Rust program to echo lines from stdin is slow?

Submitted by: @import:stackexchange-codereview··
0
Viewed 0 times
thiscanwhyhowstdinprogramechoslowfindrust

Problem

Given the following Rust program:

fn main() {
    let mut reader = io::stdin();
    for line in reader.lock().lines() {
        match line {
            Ok(l) => print!("{}", l),
            Err(_) => continue,
        }
    }
}


yes | program achieves 1.04MiB/s throughput, according to pv. The trivial C program below, which I fully realize does less, gives me a throughput of 141MiB/s.

int main() {
    int c;
    while((c=getchar()) != EOF)
        putchar(c);
}


How do I go about finding out what's keeping the Rust version from being faster? While I would appreciate if you just told me what to change to make it faster, I'm far more interested in how to find out what needs to be changed.

Edit: I tried changing lines() to chars() in the Rust version, to approximate the C version better, but it didn't seem to make any difference.

Solution

I'm going to treat this as a code review question where the goal is to figure out how to write high-performance I/O code in Rust.

There are two critical steps to speed up Rust I/O:

  • Make sure the optimizer is turned on. Rust loops have terrible performance in debug mode. Turning the optimizer on should probably get you near 25 MB/sec or so, at least in my experience.



  • Avoid line-based I/O, or anything else which uses String values. For maximum performance, we want to work with raw byte buffers and avoid ever letting Rust call malloc or free.



Here's some code which drills down to fill_buf, which is about as low as we can go before losing generality:

use std::io;
use std::io::{IoError, IoErrorKind};

fn main() {
    let mut reader = io::stdin();
    let mut buffer = reader.lock();
    let mut writer = io::stdout();
    loop {
        let consumed = match buffer.fill_buf() {
            Ok(bytes) => { writer.write(bytes).unwrap(); bytes.len() },
            Err(IoError{kind: IoErrorKind::EndOfFile, ..}) => break,
            Err(ref err) => panic!("Failed with: {}", err)
        };
        buffer.consume(consumed);
    }
}


Of course, this means that you need to find the line breaks yourself, and be prepared for lines to be split over multiple buffers. Also, notice how I pass bytes.len() outside of the match block before trying to call buffer.consume(). This is necessary to placate the lifetime checker.

Trying this with pv:

cat /dev/zero | target/release/throughput | pv > /dev/null


…gives us a throughput of 527 MB/s.

Code Snippets

use std::io;
use std::io::{IoError, IoErrorKind};

fn main() {
    let mut reader = io::stdin();
    let mut buffer = reader.lock();
    let mut writer = io::stdout();
    loop {
        let consumed = match buffer.fill_buf() {
            Ok(bytes) => { writer.write(bytes).unwrap(); bytes.len() },
            Err(IoError{kind: IoErrorKind::EndOfFile, ..}) => break,
            Err(ref err) => panic!("Failed with: {}", err)
        };
        buffer.consume(consumed);
    }
}
cat /dev/zero | target/release/throughput | pv > /dev/null

Context

StackExchange Code Review Q#73753, answer score: 15

Revisions (0)

No revisions yet.