HiveBrain v1.2.0
Get Started
← Back to all entries
patterncppMinor

Threaded generalised transform

Submitted by: @import:stackexchange-codereview··
0
Viewed 0 times
transformthreadedgeneralised

Problem

I'm writing an application which works with huge amounts of sequential data, and often found the need to use std::transform. I see two potential improvements to std::transform:

  • Allow for variable number parameters.



  • Take advantage of the linear separability of the data by multithreading.



Can anyone suggest any design/performance improvements on my implementation?

threaded_transform.h

#include 
#include 

template
OutputIterator
trans(InputIterator first, InputIterator last, OutputIterator result,
          Function f, Params... params)
{
    for (; first != last; ++first, ++result)
        *result = f(*first, params...);
    return result;
}

template
OutputIterator
threaded_transform(unsigned num_threads, InputIterator first,
                   InputIterator last, OutputIterator result,
                   Function f, Params... params)
{
    std::size_t num_values = last - first;
    std::size_t num_values_per_threads = num_values / num_threads;

    std::vector threads;
    threads.reserve(num_threads);

    for (unsigned i = 1; i ,
                                          first, last, result, f, params...));
        } else {
            threads.push_back(std::thread(trans,
                                          first, first + num_values_per_threads, result, f, params...));
        }
        first  += num_values_per_threads;
        result += num_values_per_threads;
    }

    for (auto& thread : threads)
        thread.join();

    return result;
}


main.cpp

#include 
#include "threaded_transform.h"

int main()
{
    auto sum = [] (int a, int b) { return a + b; };

    std::vector values = {1,2,3,4,5,6,7,8,9,10};
    std::vector results;
    results.resize(10);

    threaded_transform(4, values.cbegin(), values.cend(), results.begin(), sum, 10);

    for (auto result : results) {
        std::cout << result << std::endl;
    }
}

Solution

You probably want to forward your parameters:

*result = f(*first, params...);


Try:

*result = f(*first, std::forward(params)...);


To go along with forwardign you probably want two versions of trans() on that takes values by reference/value one that takes r-value references:

// Normal parameters.
trans(InputIterator first, InputIterator last, OutputIterator result,
      Function f, Params const&... params)

// R-Value parameters.
trans(InputIterator first, InputIterator last, OutputIterator result,
      Function f, Params&&... params)


Going to main transform function.

I am not sure you in realty want to pass a function and arguments. That's the whole point of the lambda. So you can wrap the function call and its parameters into a function.

threaded_transform(4, values.cbegin(), values.cend(), results.begin(), sum, 10);

// Or would you prefer:

threaded_transform(4, values.cbegin(), values.cend(), results.begin(),
    [](int other){ return add(other, 10);}
);

//Or even
threaded_transform(4, values.cbegin(), values.cend(), results.begin(),
    [](int other){ return 10 + other;}
);


If you do this you should write details about your iterator requirements.

The requirements for std::transform()

template
OutputIterator transform(InputIterator1 first1, InputIterator1 last1,
                         InputIterator2 first2, OutputIterator result,
                         BinaryFunction binary_op);

Where:
   InputIterator must be a model of Input Iterator.
   OutputIterator must be a model of Output Iterator.


In your threaded implementation you have a more stringent requirement for the output iterator. I believe it needs to be random access iterator.

OutputIterator must be a model of Random Access Iterator.


See: http://www.sgi.com/tech/stl/RandomAccessIterator.html

Code Snippets

*result = f(*first, params...);
*result = f(*first, std::forward<Params>(params)...);
// Normal parameters.
trans(InputIterator first, InputIterator last, OutputIterator result,
      Function f, Params const&... params)

// R-Value parameters.
trans(InputIterator first, InputIterator last, OutputIterator result,
      Function f, Params&&... params)
threaded_transform(4, values.cbegin(), values.cend(), results.begin(), sum, 10);

// Or would you prefer:

threaded_transform(4, values.cbegin(), values.cend(), results.begin(),
    [](int other){ return add(other, 10);}
);

//Or even
threaded_transform(4, values.cbegin(), values.cend(), results.begin(),
    [](int other){ return 10 + other;}
);
template<class InputIterator1, class InputIterator2, class OutputIterator, class BinaryFunction>
OutputIterator transform(InputIterator1 first1, InputIterator1 last1,
                         InputIterator2 first2, OutputIterator result,
                         BinaryFunction binary_op);


Where:
   InputIterator must be a model of Input Iterator.
   OutputIterator must be a model of Output Iterator.

Context

StackExchange Code Review Q#68313, answer score: 3

Revisions (0)

No revisions yet.