Saturday 16 January 2016

Async, MySQL and connection limits

I'm using Bookshelf.js as ORM, using MySQL as database and async as async library.The following code only adds 1747 records into my database even though there are 2000 results in the array. Actually I use request library and fetch some data from a website, but I simplified it here, even if I remove the request part, I am not able to add more than 1747 records when I type curl http://localhost/async on the terminal and check the number of records in the db after a few seconds.What could be my problem? I edited my.cnf and increased max_allowed_packet like below.[mysqld] max_allowed_packet = 6400M Node.jsvar person = require('./models').person; app.get('/asynctest', function(req, res) { var people = []; for (var a = 18000; a < 20000; a++) { people.push("random url"); } async.mapLimit(people, 20, function(url, callback) { new person({ name: "YES", url: url }).save(); callback(); }); }); Here's my db.jsvar knex = require('knex')({ client: 'mysql', connection: { host: '127.0.0.1', port: 8889, user: 'root', password: 'root', database: 'mydbname', charset: 'utf8' }, pool: { min: 0, max: 100 } }); var db = require('bookshelf')(knex); module.exports = db; models.jsvar db = require('./db'); var person = db.Model.extend({ tableName: 'people' }); module.exports = { person : person };

Submitted January 16, 2016 at 09:48AM by laraveling

No comments:

Post a Comment