patternMinor
A data structure for an allocation-free dynamic sample rate buffer
Viewed 0 times
samplefreerateforstructuredynamicallocationdatabuffer
Problem
I'm looking for a data structure that would allow storing samples (in O(1)) from a data stream in a fixed-size buffer while the stream length isn't known in advance.
Once the buffer size is exhausted it should start overwriting previous elements in a staggered manner, retroactively reducing the sampling rate of the data in the buffer.
I've already come up with the algorithm, but want to know if there's a proper name for this so I can look for ready-made implementations.
Basically, for a 16-element buffer, the insertion order would be:
Basically, after storing the initial N elements, with each following M * N/2 elements stored, the buffer contains a (shuffled) history with an effective sampling rate of 1/(M+1).
Once the buffer size is exhausted it should start overwriting previous elements in a staggered manner, retroactively reducing the sampling rate of the data in the buffer.
I've already come up with the algorithm, but want to know if there's a proper name for this so I can look for ready-made implementations.
Basically, for a 16-element buffer, the insertion order would be:
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
16 18 20 22 24 26 28 30 [a]
32 36 40 44
48 52 56 60
64 72
80 88
96 104
112 120
128
144
160
176
192
208
224
240
Sorted buffer contents after the first 16 insertions:
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Sorted buffer contents after the first 32 insertions:
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30
After 64 insertions:
0 4 8 12 16 20 24 28 32 36 40 44 48 52 56 60
...and so on.Basically, after storing the initial N elements, with each following M * N/2 elements stored, the buffer contains a (shuffled) history with an effective sampling rate of 1/(M+1).
Solution
The best name I can think of is derandomized or deterministic reservoir sampling. I don't know of any ready-made implementations.
Context
StackExchange Computer Science Q#165127, answer score: 4
Revisions (0)
No revisions yet.