Monday 24 February 2020

JavaScript heap out of memory

Hello!I'm using mysql2 with a transaction to insert 1,000,000 of rows and node crashes with a JavaScript heap out of memory.The records are simple, an integer, a date, and four doubles, about of 39mb of data for 1,000,000 of rows.This is the code inside the transaction:await arr.map(async (row, index) => {await conn.execute(sql, row).catch(e => {log.index = index;log.row = row;log.errorMessage = e.message;logError(log_id, log);throw new Exception('Error inserting data');});});to I'm starting node with --max-old-space-size=4096"scripts": {"dev": "cross-env NODE_OPTIONS=\"--max-old-space-size=4096\" nodemon index.js"},Is there a better way to deal with this situation? Can I flush the GC before run the transaction? If so, how?Increase the --max-old-space-size isn't an option, I'm looking for a way to optimize.It could be nice to be able at least log the error in the database, but if node is already crashed... :/​Thanks in advance!

Submitted February 25, 2020 at 03:27AM by jjvgarcia

No comments:

Post a Comment