snippetjavaMinor
Create an object based on data from a text file
Viewed 0 times
filecreatetextobjectbasedfromdata
Problem
The following method reads specially formatted text (TOML) from a file and parses the input using a parser to get a few properties, which are then used to create an object.
The profiler tells me that this method is the bottleneck in my program. While parsing ~5000 files, this method eats up over 95% of the total running time. While in another method, I am able to write the data parsed from this method to a text file in under 4 seconds for more than 5000 files, this method takes around 15 seconds on an average to run.
```
/**
* Creates an appropriate instance of a Parsable implementation depending
* upon the header of the file.
*
* @param file the path of the file from which to create a Parsable.
* @return the created Parsable.
*/
private Parsable createParsable(Path file) {
Toml toml = new Toml();
try (BufferedReader br = Files.newBufferedReader(file, StandardCharsets.UTF_8)) {
StringBuilder header = new StringBuilder();
String line;
while ((line = br.readLine()) != null && !line.equals(HEADER_DELIMITER)) {
header.append(line).append("\n");
}
toml.parse(header.toString());
String title = toml.getString("title");
author = toml.getString("author") != null ? toml.getString("author") : author;
String date = toml.getString("date");
String slug = toml.getString("slug");
LocalDate publishDate = LocalDate.parse(date, DateTimeFormatter.ofPattern(config.getDateFormat()));
String layout = toml.getString("layout");
List tag = toml.getList("tags");
StringBuilder content = new StringBuilder();
while ((line = br.readLine()) != null) {
content.append(line).append("\n");
}
if (layout.equals("post")) {
return new Post(title, author, publishDate, file, content.toString(), slug, layout, tag);
}
else {
return new Page(title, author, file, content.toString(), slug, layout, tag);
}
The profiler tells me that this method is the bottleneck in my program. While parsing ~5000 files, this method eats up over 95% of the total running time. While in another method, I am able to write the data parsed from this method to a text file in under 4 seconds for more than 5000 files, this method takes around 15 seconds on an average to run.
```
/**
* Creates an appropriate instance of a Parsable implementation depending
* upon the header of the file.
*
* @param file the path of the file from which to create a Parsable.
* @return the created Parsable.
*/
private Parsable createParsable(Path file) {
Toml toml = new Toml();
try (BufferedReader br = Files.newBufferedReader(file, StandardCharsets.UTF_8)) {
StringBuilder header = new StringBuilder();
String line;
while ((line = br.readLine()) != null && !line.equals(HEADER_DELIMITER)) {
header.append(line).append("\n");
}
toml.parse(header.toString());
String title = toml.getString("title");
author = toml.getString("author") != null ? toml.getString("author") : author;
String date = toml.getString("date");
String slug = toml.getString("slug");
LocalDate publishDate = LocalDate.parse(date, DateTimeFormatter.ofPattern(config.getDateFormat()));
String layout = toml.getString("layout");
List tag = toml.getList("tags");
StringBuilder content = new StringBuilder();
while ((line = br.readLine()) != null) {
content.append(line).append("\n");
}
if (layout.equals("post")) {
return new Post(title, author, publishDate, file, content.toString(), slug, layout, tag);
}
else {
return new Page(title, author, file, content.toString(), slug, layout, tag);
}
Solution
I see that the processing you're doing here does not depend on the order you're processing in. This means you can parallelize the processing heavily.
Additionally you're doing line-by-line processing, which allows you to use one of the new features of Java 8, namely
This greatly simplifies the code you have to following outline:
Also it might be faster to keep parsables in "one" file, this reduces channel and OS waiting overhead for Open/Close operations when doing I/O
Additionally you're doing line-by-line processing, which allows you to use one of the new features of Java 8, namely
Files.linesThis greatly simplifies the code you have to following outline:
try (Stream lines = Files.lines(path).parallel()) {
// do line by line processing
} catch (IOException e) {
// sensible handling
}Also it might be faster to keep parsables in "one" file, this reduces channel and OS waiting overhead for Open/Close operations when doing I/O
Code Snippets
try (Stream<String> lines = Files.lines(path).parallel()) {
// do line by line processing
} catch (IOException e) {
// sensible handling
}Context
StackExchange Code Review Q#95535, answer score: 2
Revisions (0)
No revisions yet.