HiveBrain v1.2.0
Get Started
← Back to all entries
snippetsqlMinor

How to check if all elements of an array exists in a table?

Submitted by: @import:stackexchange-dba··
0
Viewed 0 times
elementsallarrayexistshowchecktable

Problem

Lets take an array of ids of size n in the application level. for instance [132,3425,13,13,... 392] where n is bigger than 100k entries. Some of this entries are repeated.

I need to check if all of these registers are contained in a MySQL table containing more than 5M entries.

Now I'm checking one after another, but this process takes a very long time.

How can I make this verification in one single check, or maybe make this more efficient.

I would like to know if all of them exists on the table, and, if possible, know which don't already exists on that table.

Solution

CREATE TEMPORARY TABLE B_Temp (
     id BIGINT NOT NULL
     PRIMARY KEY (id)
     INDEX unid (id)
);


I would load a temp table as in @AMG 's answer. But my query would look like this:

select id
from a
where not exists (
select *
from B_Temp
where a.id = b.id
)


You would want to disable the index on the id column before you load the data, and re-enable it after to get this to perform well.
If you are doing this on a regular basis, you might consider making this a permanent table.

Code Snippets

CREATE TEMPORARY TABLE B_Temp (
     id BIGINT NOT NULL
     PRIMARY KEY (id)
     INDEX unid (id)
);
select id
from a
where not exists (
select *
from B_Temp
where a.id = b.id
)

Context

StackExchange Database Administrators Q#211355, answer score: 4

Revisions (0)

No revisions yet.