Re: Help with large database import

Hi Matthew,
  I have configured few of the Cluster variables. I am listing down some of them:

datadir=/vmware/data/ # Directory for this data node's data files MaxNoOfConcurrentOperations=159360
MaxNoOfConcurrentTransactions=159360
MaxNoOfConcurrentIndexOperations=159360
TimeBetweenLocalCheckpoints=9
NoOfFragmentLogFiles=300
DataMemory=2000M
IndexMemory=800M
RedoBuffer=32M

now i have a different issue.

I have 13GB of disk partition. The DB size is around 2GB in MyISAM. After importing several times the DB has grown in CLUSTER and occupied up to 13 GB. The disk is full. What are the log files i can clear. I have given NoOfFragmentLogFiles=300 is that the concern here?

Thanks for the response.

regards,
Pradeep

On 10/15/07, Matthew Montgomery <mmontgomery@mysql.com> wrote:
>
> Hello Pradeep,
>
> On Mon, 2007-10-15 at 13:26 +0530, Pradeep Chandru wrote:
> > Hi,
> > I have the similar kind of issue in MySQL 5.0.45.
> >
> > ERROR 1114 (HY000) at line 1702: The table 'qp_weightage_1' is full
> >
> >
> > I am trying the installation of mysql cluster for the first time.
> > I am trying to import a db of size 900MB in to the cluster. The server
> has
> > got 8GB RAM. I have not configured any of the parameters like data
> memory,
> > index memory... etc.
> > I have the following doubts:
>
>
>
> >
> > 1. How can i load a huge table. What are the variables to be tuned?
> > what are
> > the parameters to be considered for the import to be successful?
>
> You *must* first define your DataMemory and IndexMemory. The default
> values for DataMemory is 80M and 18M for IndexMemory.
>
> Optionally you should make sure you have sufficient MaxNoOfTables,
> MaxNoOfAttributes, MaxNoOfOrderedIndexes and MaxNoOfUniqueHashIndexes
>
> You should also know that tables in cluster can require far more room to
> store the tables than in other engines. This is especially true in 5.0
> because VARCHAR columns are stored in memory with a fixed length. So a
> VARCHAR(255) column will require a minimum of 255 bytes + 16 bytes
> overhead of DataMemory per row. VARCHAR columns can require as much as
> three times this amount for UTF-8 characters.
>
> > 2. Is there any link where i can find more data on tuning the variables.
> >
>
> You can use the ndb_size.pl script to determine the recommended values
> for IndexMemory and DataMemory.
>
> See also:
> http://dev.mysql.com/doc/refman/5.0/en/mysql-cluster-ndbd-definition.html
>
> http://dev.mysql.com/doc/refman/5.0/en/mysql-cluster-config-params-ndbd.html
>
>
> --
> Matthew Montgomery, Support Engineer
> San Antonio, TX
> MySQL Inc, www.mysql.com
>
>
Received on Wed Oct 17 01:57:09 2007