HiveBrain v1.2.0
Get Started
← Back to all entries
patternMinor

A data structure for an allocation-free dynamic sample rate buffer

Submitted by: @import:stackexchange-cs··
0
Viewed 0 times
samplefreerateforstructuredynamicallocationdatabuffer

Problem

I'm looking for a data structure that would allow storing samples (in O(1)) from a data stream in a fixed-size buffer while the stream length isn't known in advance.

Once the buffer size is exhausted it should start overwriting previous elements in a staggered manner, retroactively reducing the sampling rate of the data in the buffer.

I've already come up with the algorithm, but want to know if there's a proper name for this so I can look for ready-made implementations.

Basically, for a 16-element buffer, the insertion order would be:

0  1  2  3  4  5  6  7  8  9  10 11 12 13 14 15
   16    18    20    22    24    26    28    30 [a]
      32          36          40          44
         48          52          56          60
            64                      72
               80                      88
                  96                      104
                     112                     120
                        128
                           144
                              160
                                 176
                                    192
                                       208
                                          224
                                             240

Sorted buffer contents after the first 16 insertions:
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Sorted buffer contents after the first 32 insertions:
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30

After 64 insertions:
0 4 8 12 16 20 24 28 32 36 40 44 48 52 56 60

...and so on.


Basically, after storing the initial N elements, with each following M * N/2 elements stored, the buffer contains a (shuffled) history with an effective sampling rate of 1/(M+1).

Solution

The best name I can think of is derandomized or deterministic reservoir sampling. I don't know of any ready-made implementations.

Context

StackExchange Computer Science Q#165127, answer score: 4

Revisions (0)

No revisions yet.