patternsqlModerate
Possible to keep max number of records in postgresql?
Viewed 0 times
postgresqlnumberrecordskeeppossiblemax
Problem
Basically part of our Postgresql table is used to keep server access logs, and as such sometimes during production this can get pretty large. is there any way of setting in postgresql to have a maximum number of records a table can have and to push off the oldest record?
Solution
You can define a trigger to maintain your desired row number:
This is probably not the best performing option, but once you reach the limit, it will never be exceeded. If there is space for fluctuation, then you can check the row number periodically and delete excess rows from the beginning.
EDIT:
If you have really large logs (say a million per month) than partitioning can be the easiest solution. You can then simply drop the unnecessary tables (say where
But be careful before discarding old logs. Are you sure you will never need those?
CREATE OR REPLACE FUNCTION trf_keep_row_number_steady()
RETURNS TRIGGER AS
$body$
BEGIN
-- delete only where are too many rows
IF (SELECT count(id) FROM log_table) > rownum_limit
THEN
-- I assume here that id is an auto-incremented value in log_table
DELETE FROM log_table
WHERE id = (SELECT min(id) FROM log_table);
END IF;
END;
$body$
LANGUAGE plpgsql;
CREATE TRIGGER tr_keep_row_number_steady
AFTER INSERT ON log_table
FOR EACH ROW EXECUTE PROCEDURE trf_keep_row_number_steady();This is probably not the best performing option, but once you reach the limit, it will never be exceeded. If there is space for fluctuation, then you can check the row number periodically and delete excess rows from the beginning.
EDIT:
If you have really large logs (say a million per month) than partitioning can be the easiest solution. You can then simply drop the unnecessary tables (say where
max(timestamp) < CURRENT_DATE - 1 year). You can use your timestamp (or a derived date) as condition for range partitioning.But be careful before discarding old logs. Are you sure you will never need those?
Code Snippets
CREATE OR REPLACE FUNCTION trf_keep_row_number_steady()
RETURNS TRIGGER AS
$body$
BEGIN
-- delete only where are too many rows
IF (SELECT count(id) FROM log_table) > rownum_limit
THEN
-- I assume here that id is an auto-incremented value in log_table
DELETE FROM log_table
WHERE id = (SELECT min(id) FROM log_table);
END IF;
END;
$body$
LANGUAGE plpgsql;
CREATE TRIGGER tr_keep_row_number_steady
AFTER INSERT ON log_table
FOR EACH ROW EXECUTE PROCEDURE trf_keep_row_number_steady();Context
StackExchange Database Administrators Q#20145, answer score: 12
Revisions (0)
No revisions yet.