I'm not sure what's the best way to deal with streams that need to be closed whenever I'm done using them? Usually the need to also catch the exception inside the `finally` statement is very, very annoying. Am I doing this right? ``` public class FileSender { public static final String KEY = "00000000000000000000000000000000"; public static final String ALGORITHM = "AES"; public static final String FILE = "C:\\Users\\Andres\\Desktop\\file.txt"; public static final String HASH_ALG = "MD5"; private String filePath; private String serverMachine; private int serverPort; public FileSender(String filePath, String serverMachine, int serverPort){ this.filePath = filePath; this.serverMachine = serverMachine; this.serverPort = serverPort; } public void sendFile(){ CipherOutputStream cipherOut = null; Socket client = null; String checkSum; try { System.out.println("Running client ..."); client = connectToServer(); checkSum = getHashChecksum(); cipherOut = initCipherStream(client); transferData(cipherOut, checkSum); } catch (Exception e) { System.out.println(e.getMessage()); } finally { } } private String getHashChecksum() { FileInputStream fis = null; try { MessageDigest md = MessageDigest.getInstance(HASH_ALG); fis = new FileInputStream(this.filePath); byte[] byteArray = new byte[1024]; int bytesCount = 0; while ((bytesCount = fis.read(byteArray)) >= 0) md.update(byteArray, 0, bytesCount); byte[] hash = md.digest(); fis.close(); return String.format("%032x",new BigInteger(1, hash)); } catch (Exception e) { e.printStackTrace(); throw new RuntimeException("Error in checksum"); } finally { try {

I have three classes: product, customer and application. The schema of these classes are as follows: ``` class application { String productID; String customerID; String applicationId; String name; .... // getters and setters and associated constructors } class product { String name; String ID; ... // constructors and getters and setters } class customer { String name; String ID; ... // constructors and getters and setters } ``` The only way of connection between product and customer is through the application. The problem in hand is to retrieve all products with their associated customers (appID and customerID) in this JSON format. ``` [ { "product": { // product details } "customers": [ { "application": , "ID": } ] } ] ``` I have done this using a nested for loop: ``` for (Product product : products) { ProductInfo prodInfo = new ProductInfo(); prodInfo.setProduct(product); List associatedCustomerList = new ArrayList<>(); for (Object res : searchResults.getResult()) { if ((product.getId()).equals((String) ((Map) res).get("productID"))) { CustomerInfo associatedCustomer = new CustomerInfo(); associatedCustomer.setCustomerID((String) ((Map) res).get("customerID")); associatedCustomer.setApplicationId((String) ((Map) res).get("appId")); associatedCustomerList.add(associatedCustomer); } prodInfo.setCustomers(associatedCustomerList); } productInfo.add(prodInfo); } ``` Here, `searchResults` contain the result from `List`. `CustomerDetails` contain `appId` and `customerId` as attributes. `ProductInformationWithAssociatedCustomer` is the response class. Furthermore, I have converted this into the following which utilizes Java 8 Stream API: ``` List prodList = products.stream() .map(product -> { List customerList =searchResults.getResult().stream()

I have the following given code after some refactoring: ``` public static void storePreferences() { try { Preferences.userRoot().exportSubtree(new FileOutputStream(ReleaseInfo.getAppFolder() + File.separator + SETTINGSFILENAME)); } catch (IOException e) { e.printStackTrace(); } catch (BackingStoreException e) { e.printStackTrace(); } } ``` Now I wonder - should I explicitly close the `FileOutputStream` or the JVM takes care for doing so? I have looked at this question on the same topic, where it is said that it is not a good practice to rely on the garbage collector, but still I need a bit more confirmation (therefore this question). I have changed the above code to: ``` public static void storePreferences() { FileOutputStream fos = null; try { fos = new FileOutputStream(ReleaseInfo.getAppFolder() + File.separator + SETTINGSFILENAME); Preferences.userRoot().exportSubtree(fos); } catch (IOException | BackingStoreException e) { e.printStackTrace(); } finally { try { fos.close(); } catch (IOException e) { e.printStackTrace(); } } } ``` Here I close the stream manually, so which would be the better alternative and do the performed changes really bring something to the table? Any other remarks?

Have a working version of MySQL implementation of ThorsSQL library done. If you want to check it out you can find the whole thing on github ThorsSQL. This is a follow on to previous code Reviews: Part 2 Part 1 The documentation for these classes is here: Part 3 (Layer 1): The MySQL Implementation Layer 1: Simple Stream The lowest layer is the stream. This is simply an open bidirectional unix socket to the server. This is implemented by the `MySQLStream` class which implements the Interface `PackageStream`. By providing this layer of abstraction I can mock out the stream object for testing; thus allowing the testing framework to replay with a specific stream of bytes without actually needing to use a socket. ``` Class: ========== MySQLStream: PackageStream Interface: ========== PackageStream ``` PackageStream.h ``` #ifndef THORS_ANVIL_MYSQL_PACKAGE_STREAM_H #define THORS_ANVIL_MYSQL_PACKAGE_STREAM_H #include #include namespace ThorsAnvil { namespace MySQL { class PackageStream { public: virtual ~PackageStream() = 0; virtual void read(char* buffer, std::size_t len) = 0; virtual void write(char const* buffer, std::size_t len) = 0; virtual void startNewConversation() = 0; virtual void flush() = 0; virtual void reset() = 0; virtual void drop() = 0; virtual bool isEmpty() = 0; virtual std::string readRemainingData() = 0; }; } } #endif ``` MySQLStream.h ``` #ifndef THORS_ANVIL_MYSQL_MY_SQL_STREAM_H #define THORS_ANVIL_MYSQL_MY_SQL_STREAM_H #include "PackageStream.h" #include #include namespace ThorsAnvil { namespace MySQL { class MySQLStream: public PackageStream { s

I've been using Java 8 Stream features quite a lot these days and am very happy with it. I've written a code that uses groupingBy and max but was wondering if it can still be optimized or even just be tweaked for better readability. Some sample data: ``` Name, OrderDate, DeliveryTime abc, 2010-01-15, 2010-01-15T11:00:00 abc, 2010-01-15, 2010-01-31T07:00:00 //should be marked as latest zzz, 2010-01-15, 2010-01-13T11:00:00 zzz, 2010-01-15, 2010-01-23T07:00:00 //should be marked as latest ``` Here's the code I've written: ``` public class DeliveryTest { class Delivery { String name; LocalDate orderDate; LocalDateTime deliveryTime; boolean isLatest = false; //getters and setters here } @Test public void testLatestDelivery() { List deliveries = new ArrayList<>(); //populate the test data here //group the deliveries by name and orderDate Map>> groupByNameAndOrderDate = deliveries.stream() .collect(groupingBy(Delivery::getName, groupingBy(Delivery::getOrderDate))); //find and mark as "Latest" for each of the the max deliveryTime per name and orderDate grouping, groupByNameAndOrderDate.forEach((name, byNameMap) -> { byNameMap.forEach((orderDate, byOrderDateList) -> { byOrderDateList.stream() .max((n1, n2) -> n1.getDeliveryTime().compareTo(n2.getDeliveryTime())).get() .setIsLatest(true); }); }); //do assertions here } } ``` Can this Java 8 code still be improved (for example the part where I have nested forEach)? Or written shorter but still readable (for example, can the group and max be combined)?

Summary: I am using an API which returns paged results. I want to have these results as Java 8 `Stream` and implemented a `Spliterator` for this purpose. I am using AWS S3 Java API to list objects in the S3 bucket. The API returns paged results: when I call `client.listObjects(bucketName, rootKey)` for the first time, I get an `ObjectListing` instance which may be complete or truncated i.e. return just one "page" of results. If `ObjectListing` is truncated I have to request further "page" via `client.listNextBatchOfObjects(objectListing)` (providing current "page" as marker) and so on until I get an `ObjectListing` which is not truncated. I want to use Java 8 `Stream` APIs to work with `ObjectListing`s. Ideally, I want to hide querying pages of `ObjectListing`s behind some facility which would just give me a `Stream`. For this I've implemented a `Spliterator`: ``` import java.util.Objects; import java.util.Spliterator; import java.util.function.Consumer; import com.amazonaws.services.s3.AmazonS3; import com.amazonaws.services.s3.model.ObjectListing; public class ObjectListingSpliterator implements Spliterator { private final AmazonS3 client; private ObjectListing objectListing; private volatile boolean split = false; private volatile boolean currentObjectListingWasConsumed = false; public ObjectListingSpliterator(AmazonS3 client, ObjectListing objectListing) { Objects.requireNonNull(client, "client must not be null."); Objects.requireNonNull(objectListing, "objectListing must not be null."); this.client = client; this.objectListing = objectListing; } @Override public boolean tryAdvance(Consumer action) { if (!currentObjectListingWasConsumed) { action.accept(objectListing); currentObjectListingWasConsumed = true; if (!split && objectListing.isTruncated()) { objectListing = client.listNextBatchOfObjects(objectListing); cu

I wrote the following as more of an experiment than anything else, but I thought it would be fun to share and maybe get some feedback! Motivation: I started looking at some functional languages and noticed how useful the `|>` pipeline operator is. I started wondering how it might be translated to the Java language. There are many common functions omitted from the `Stream` library that could be useful in expressing logic in a more clear fashion. For example, `Stream` does not have an instance method to `concat` with another stream-- you have to use a static method. This means that you can't easily chain `concat` to another operation within a stream; you have to wrap it within a method call. More examples include reversing, delays, zipping, etc. For each of these operations you have to make a helper method, and just like with `concat` you have to break chaining to wrap the partial solution within a method. This starts getting messy quickly, preventing authors from writing readable one-line expressions. Consider the following example where static and instance methods are called on a stream: ``` List foo = Stream .concat( reverseStream( list1.stream() .map(Some::func) .flatMap(other::stuff)), list2.stream() .map(Some::func) .flatMap(other::stuff))) .map(Some::otherFunc) .collect(Collectors.toList()) ... ``` With a pipeline `|>` operator, you could hide some complexity: ``` List foo = list1 |> stream() |> map(Some::func) |> flatMap(other::stuff)) |> reverseStream() |> concat(list1 |> stream() |> map(Some::func) |> flatMap(other::stuff)) |> map(Some::otherFunc) |> collect(Collectors.toList()); ``` The following are the methods I wrote which enable piping `t |> f`: ``` /** * Perform a pipe operation on a parameter and a function. * This follows the form: R = T |> F * * @param t The parameter to be applied * @param f The fun

An API (I have no control over) gives me a bunch of `Collection`s (`List`s, to be precise,) and I need the elements that are common to all of them. In this concrete case, I'm trying to find Selenium `WebElement`s that match several selectors. But I went off on a Yak-Shaving spree and tried to figure out how to make an intersection of a stream of `Collection`s. This is what I came up with (and I think it works, at least my tests say so.) ``` /** * Given a stream of collections, intersect it. * * @param stream A stream of collections you want to intersect * @param Any item that has a decent .equals()/.hashCode(). * Seriously, don't attempt this without a working .equals()/.hashCode(). * @return The unique(!) elements present in *all* collections in the stream */ public static Collection intersect(Stream> stream) { // Optimization: sorting by size so that the biggest constrainer // (smallest collection) comes first final Iterator> allLists = stream.sorted( (l1, l2) -> l1.size() - l2.size() ).iterator(); final Set result = new HashSet<>(allLists.next()); while(allLists.hasNext()) { result.retainAll(allLists.next()); } return result; } ``` I fiddled around with `Collector`s in the beginning, but that seemed to be too convoluted, since it'd require the unit element to be some special class. Any better way of doing it? Is this woefully inefficient? I think its complexity is somewhere around O(n) where n should be the total number of elements in all the collections.

This is part of the Construct library. Rebuffered is used to turn a seekless/tellless stream like from a socket or pipe into a seekable/tellable one like a BytesIO or a file. If `tailcutoff` is provided, only that amount of bytes is kept counting from the end of last read write operation backwards. Otherwise entire stream is cached. ``` class RebufferedBytesIO: __slots__ = ["substream","offset","rwbuffer","moved","tailcutoff"] def __init__(self, substream, tailcutoff=None): self.substream = substream self.offset = 0 self.rwbuffer = b"" self.moved = 0 self.tailcutoff = tailcutoff def read(self, count): startsat = self.offset endsat = startsat + count if startsat < self.moved: raise IOError("could not read because tail was cut off") while self.moved + len(self.rwbuffer) < endsat: newdata = self.substream.read(128*1024) self.rwbuffer += newdata if not newdata: sleep(0) data = self.rwbuffer[startsat-self.moved:endsat-self.moved] self.offset += count if self.tailcutoff is not None and self.moved < self.offset - self.tailcutoff: removed = self.offset - self.tailcutoff - self.moved self.moved += removed self.rwbuffer = self.rwbuffer[removed:] if len(data) < count: raise IOError("could not read enough bytes, something went wrong") return data def write(self, data): startsat = self.offset endsat = startsat + len(data) while self.moved + len(self.rwbuffer) < startsat: newdata = self.substream.read(128*1024) self.rwbuffer += newdata if not newdata: sleep(0) self.rwbuffer = self.rwbuffer[:startsat-self.moved] + data + self.rwbuffer[endsat-self.moved:] self.offset = endsat if self.tailcutoff is not None and self.moved < self.offset - self.

There are a plethora of streams and streambuffers available today, none are as fast as a memory `streambuf`, that you can use for testing. Shoot away! ``` #ifndef MEMSTREAMBUF_HPP # define MEMSTREAMBUF_HPP # pragma once #include #include #include #include #include template class memstreambuf : public ::std::streambuf { ::std::array buf_; public: memstreambuf() { setbuf(buf_.data(), buf_.size()); } ::std::streambuf* setbuf(char_type* const s, ::std::streamsize const n) final { auto const begin(s); auto const end(s + n); setg(begin, begin, end); setp(begin, end); return this; } pos_type seekpos(pos_type const pos, ::std::ios_base::openmode const which = ::std::ios_base::in | ::std::ios_base::out) final { switch (which) { case ::std::ios_base::in: if (pos class memstream : public memstreambuf, public ::std::istream, public ::std::ostream { public: memstream() : ::std::istream(this), ::std::ostream(this) { } }; #endif // MEMSTREAMBUF_HPP ```