patterncppMinor
Threaded generalised transform
Viewed 0 times
transformthreadedgeneralised
Problem
I'm writing an application which works with huge amounts of sequential data, and often found the need to use
Can anyone suggest any design/performance improvements on my implementation?
threaded_transform.h
main.cpp
std::transform. I see two potential improvements to std::transform:- Allow for variable number parameters.
- Take advantage of the linear separability of the data by multithreading.
Can anyone suggest any design/performance improvements on my implementation?
threaded_transform.h
#include
#include
template
OutputIterator
trans(InputIterator first, InputIterator last, OutputIterator result,
Function f, Params... params)
{
for (; first != last; ++first, ++result)
*result = f(*first, params...);
return result;
}
template
OutputIterator
threaded_transform(unsigned num_threads, InputIterator first,
InputIterator last, OutputIterator result,
Function f, Params... params)
{
std::size_t num_values = last - first;
std::size_t num_values_per_threads = num_values / num_threads;
std::vector threads;
threads.reserve(num_threads);
for (unsigned i = 1; i ,
first, last, result, f, params...));
} else {
threads.push_back(std::thread(trans,
first, first + num_values_per_threads, result, f, params...));
}
first += num_values_per_threads;
result += num_values_per_threads;
}
for (auto& thread : threads)
thread.join();
return result;
}main.cpp
#include
#include "threaded_transform.h"
int main()
{
auto sum = [] (int a, int b) { return a + b; };
std::vector values = {1,2,3,4,5,6,7,8,9,10};
std::vector results;
results.resize(10);
threaded_transform(4, values.cbegin(), values.cend(), results.begin(), sum, 10);
for (auto result : results) {
std::cout << result << std::endl;
}
}Solution
You probably want to forward your parameters:
Try:
To go along with forwardign you probably want two versions of
Going to main transform function.
I am not sure you in realty want to pass a function and arguments. That's the whole point of the lambda. So you can wrap the function call and its parameters into a function.
If you do this you should write details about your iterator requirements.
The requirements for
In your threaded implementation you have a more stringent requirement for the output iterator. I believe it needs to be random access iterator.
See: http://www.sgi.com/tech/stl/RandomAccessIterator.html
*result = f(*first, params...);Try:
*result = f(*first, std::forward(params)...);To go along with forwardign you probably want two versions of
trans() on that takes values by reference/value one that takes r-value references:// Normal parameters.
trans(InputIterator first, InputIterator last, OutputIterator result,
Function f, Params const&... params)
// R-Value parameters.
trans(InputIterator first, InputIterator last, OutputIterator result,
Function f, Params&&... params)Going to main transform function.
I am not sure you in realty want to pass a function and arguments. That's the whole point of the lambda. So you can wrap the function call and its parameters into a function.
threaded_transform(4, values.cbegin(), values.cend(), results.begin(), sum, 10);
// Or would you prefer:
threaded_transform(4, values.cbegin(), values.cend(), results.begin(),
[](int other){ return add(other, 10);}
);
//Or even
threaded_transform(4, values.cbegin(), values.cend(), results.begin(),
[](int other){ return 10 + other;}
);If you do this you should write details about your iterator requirements.
The requirements for
std::transform()template
OutputIterator transform(InputIterator1 first1, InputIterator1 last1,
InputIterator2 first2, OutputIterator result,
BinaryFunction binary_op);
Where:
InputIterator must be a model of Input Iterator.
OutputIterator must be a model of Output Iterator.In your threaded implementation you have a more stringent requirement for the output iterator. I believe it needs to be random access iterator.
OutputIterator must be a model of Random Access Iterator.See: http://www.sgi.com/tech/stl/RandomAccessIterator.html
Code Snippets
*result = f(*first, params...);*result = f(*first, std::forward<Params>(params)...);// Normal parameters.
trans(InputIterator first, InputIterator last, OutputIterator result,
Function f, Params const&... params)
// R-Value parameters.
trans(InputIterator first, InputIterator last, OutputIterator result,
Function f, Params&&... params)threaded_transform(4, values.cbegin(), values.cend(), results.begin(), sum, 10);
// Or would you prefer:
threaded_transform(4, values.cbegin(), values.cend(), results.begin(),
[](int other){ return add(other, 10);}
);
//Or even
threaded_transform(4, values.cbegin(), values.cend(), results.begin(),
[](int other){ return 10 + other;}
);template<class InputIterator1, class InputIterator2, class OutputIterator, class BinaryFunction>
OutputIterator transform(InputIterator1 first1, InputIterator1 last1,
InputIterator2 first2, OutputIterator result,
BinaryFunction binary_op);
Where:
InputIterator must be a model of Input Iterator.
OutputIterator must be a model of Output Iterator.Context
StackExchange Code Review Q#68313, answer score: 3
Revisions (0)
No revisions yet.