Serious OutOfMemory issue while reading with mysql-connector-j-5.0.7 ...

Hi!

Yesterday I was supprised to get an out of memory exception when reading a big table rec by rec without any caching. The program died in stmt.executeQuery(), long before I can do anything with the records.

I've tried different connection options and debugged connector/j ... every time code runs into MysqlIO.readAllResults which can not work, if the table has more records than available in the jvm. More interesting - also server side prepared statements run into that readAll-routine?

Tested with
connector-j-5.0.3 + 5.0.7,
mysql db 5.0.41 + some older 5.0 on windows.

C:\Test\src>javac -cp ..\..\lib\jdbc\mysql-connector-java-5.0.7-bin.jar MySQLRead.java
C:\Test\src>java -cp ..\..\lib\jdbc\mysql-connector-java-5.0.7-bin.jar;. MySQLRead
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

        at com.mysql.jdbc.Buffer.getBytes(Buffer.java:198)
        at com.mysql.jdbc.Buffer.readLenByteArray(Buffer.java:318)
        at 
com.mysql.jdbc.MysqlIO.extractNativeEncodedColumn(MysqlIO.java:3567)
        at 
com.mysql.jdbc.MysqlIO.unpackBinaryResultSetRow(MysqlIO.java:3483)
        at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1391)
        at com.mysql.jdbc.MysqlIO.readSingleRowSet(MysqlIO.java:2369)
        at com.mysql.jdbc.MysqlIO.getResultSet(MysqlIO.java:451)
        at 
com.mysql.jdbc.MysqlIO.readResultsForQueryOrUpdate(MysqlIO.java:2076)
        at com.mysql.jdbc.MysqlIO.readAllResults(MysqlIO.java:1451)
        at 
com.mysql.jdbc.ServerPreparedStatement.serverExecute(ServerPreparedStatement.java:1314)
        at 
com.mysql.jdbc.ServerPreparedStatement.executeInternal(ServerPreparedStatement.java:740)
        at 
com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:1448)
        at MySQLRead.main(MySQLRead.java:23)

Regards
Paul

-- 
--

_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/
_/
_/   BDM Business Data Management GmbH
_/
_/   Firmenbuch: FN 242059w
_/   Gerichtsstand: Landesgericht Wiener Neustadt  
_/   UID: ATU57685758
_/
_/   Grenzgasse 111, Objekt 9/4
_/   2340 Mödling
_/   Tel,Fax: +43 2236 46 393
_/
_/   Ing. Paul Palaszewski, (Hons) B.Sc.
_/   Geschäftsführer
_/   Mobil: +43(650) 37 36 030
_/   E-Mail: paul.palaszewski@bdm-systems.com
_/
_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/

 

import java.sql.Connection; import java.sql.DriverManager; import java.sql.PreparedStatement; import java.sql.ResultSet; /** * @author Paul Palaszewski * @since 23.08.2007 */ public class MySQLRead { public static void main(String[] args) { try { Class.forName("com.mysql.jdbc.Driver"); // tried // jdbc:mysql://localhost:3306/test?jdbcCompliantTruncation=false&zeroDateTimeBehavior=convertToNull&useCursorFetch=true // jdbc:mysql://localhost:3306/test?jdbcCompliantTruncation=false&zeroDateTimeBehavior=convertToNull // jdbc:mysql://localhost:3306/test // jdbc:mysql://localhost:3306/test?useServerPrepStmts=true Connection conn = DriverManager.getConnection("jdbc:mysql://localhost:3306/test?useServerPrepStmts=true", "root", ""); PreparedStatement stmt = conn.prepareStatement("SELECT * FROM wb_absatz"); // tried with / without fetch size stmt.setFetchSize(50); ResultSet rs = stmt.executeQuery(); long start = System.currentTimeMillis(); int recCount = 0; while (rs.next()) { rs.getString(1); rs.getString(2); rs.getString(3); recCount++; } stmt.close(); conn.close(); long duration = System.currentTimeMillis() - start; System.out.println("Read " + recCount + " records in " + (duration / 1000.0) + "s"); } catch (Exception ex) { ex.printStackTrace(); } } }

-- MySQL Java Mailing List For list archives: http://lists.mysql.com/java To unsubscribe: http://lists.mysql.com/java?unsub=lists@pantek.com Received on Thu Aug 23 05:31:24 2007

Do you need help?

In addition to our free technical library, Pantek provides professional IT services to companies who utilize Open Source software: Learn more about Pantek's Technical Support offerings Place an order for 24/7/365 Technical Support here Learn more about Pantek, who we are, and what we do Chat with a Live Operator Related lists.mysql.com Links announce benchmarks bugs certification cluster commits dotnet gui-tools internals java maxdb myodbc mysql mysqldoc packagers perl plusplus replication win32 Request more information about Pantek