[GENERAL] importing large files

From: <olivier.scalbert(at)algosyn.com>
Date: Fri Sep 28 2007 - 04:22:49 EDT

Hello,

I need to import between 100 millions to one billion records in a table. Each record is composed of two char(16) fields. Input format is a huge csv file.I am running on a linux box with 4gb of ram. First I create the table. Second I 'copy from' the cvs file. Third I create the index on the first field.
The overall process takes several hours. The cpu seems to be the limitation, not the memory or the IO.
Are there any tips to improve the speed ?

Thanks very much,

Olivier

---------------------------(end of broadcast)---------------------------

TIP 3: Have you checked our extensive FAQ?

               http://www.postgresql.org/docs/faq Received on Mon Oct 1 13:33:35 2007