snippetMinor
How to load data into Vertica from C?
Viewed 0 times
verticahowintoloadfromdata
Problem
I'm investigating column oriented databases and came across Vertica.
My need is to feed the Vertica database from C code. I don't succeed in grabbing this information from Vertica: I'm told to use "vsql" and the "copy" command. All I want is issue
Can this be done?
For instance, in PostgreSQL you can do "embedded SQL" by linking the Postgres
Any ideas?
My need is to feed the Vertica database from C code. I don't succeed in grabbing this information from Vertica: I'm told to use "vsql" and the "copy" command. All I want is issue
INSERT statements to my Vertica database.Can this be done?
For instance, in PostgreSQL you can do "embedded SQL" by linking the Postgres
ecpg library to your C binary. I have no idea if such thing exists for Vertica, and I know of no other way.Any ideas?
Solution
You have two options really:
You can create a delimited file with your data, like the following:
You can then load that into Vertica using the COPY command and vsql. If you have database superuser rights, you can load the file directly, using:
Omit -w if you want to enter keep the password out of your bash history and enter it when the program starts. If you don't have database superuser rights, you can still cat the file into vsql and get the data from STDIN - quite why Vertica prevents you from loading data from a file without superuser rights, but will let any user pipe it in via cat, I have no idea, but you can do it as follows:
Or, if you have a bunch of individual insert statements in a file, separated by semicolons, you can just run the file through vsql, as follows:
But if you're doing more than 1000 rows, bulk loading using a delimited file is supposed to be much faster.
Hope that helps!
You can create a delimited file with your data, like the following:
row1col1data,row1col2data,row1col3data,row1col4data
row2col1data,row2col2data,row2col3data,row2col4data
row3col1data,row3col2data,row3col3data,row3col4dataYou can then load that into Vertica using the COPY command and vsql. If you have database superuser rights, you can load the file directly, using:
vsql -U -w YOURDATABASENAME -c "COPY yourtablename (col1name, col2name, col3name, col4name) FROM 'yourdelimitedfile' DELIMITER ','"Omit -w if you want to enter keep the password out of your bash history and enter it when the program starts. If you don't have database superuser rights, you can still cat the file into vsql and get the data from STDIN - quite why Vertica prevents you from loading data from a file without superuser rights, but will let any user pipe it in via cat, I have no idea, but you can do it as follows:
cat yourdelimitedfile | vsql -U -w YOURDATABASENAME -c "COPY yourtablename (col1name, col2name, col3name, col4name) FROM STDIN DELIMITER ','"Or, if you have a bunch of individual insert statements in a file, separated by semicolons, you can just run the file through vsql, as follows:
vsql -U -w YOURDATABASENAME -f pathtoyoursqlfileBut if you're doing more than 1000 rows, bulk loading using a delimited file is supposed to be much faster.
Hope that helps!
Code Snippets
row1col1data,row1col2data,row1col3data,row1col4data
row2col1data,row2col2data,row2col3data,row2col4data
row3col1data,row3col2data,row3col3data,row3col4datavsql -U <username> -w <password> YOURDATABASENAME -c "COPY yourtablename (col1name, col2name, col3name, col4name) FROM 'yourdelimitedfile' DELIMITER ','"cat yourdelimitedfile | vsql -U <username> -w <password> YOURDATABASENAME -c "COPY yourtablename (col1name, col2name, col3name, col4name) FROM STDIN DELIMITER ','"vsql -U <username> -w <password> YOURDATABASENAME -f pathtoyoursqlfileContext
StackExchange Database Administrators Q#8033, answer score: 4
Revisions (0)
No revisions yet.