Hello r/nodeIt's been almost a year since I've started writing this little piece of open source software with a few friends and contributors, and at the dawn of our 2.2 release, I thought it might be a good idea to talk about it here and see what I learned during development. I also realize most apps written in NodeJS are made for the web and are intended to be deployed to servers : as you'll see, Karaoke Mugen is a standalone, client/desktop app, and some usual choices need to be reconsidered when making decisions on which technologies to use.That's why i thought it'd be an interesting post. To be honest I wanted to promote the project first, but thought providing technical details and justifying some choices made would make for a far more interesting read for r/nodeI knew nothing about NodeJS a year ago, and learned with this project, sometimes the hard way (callbacks are evil.) However, it allowed me to get a good idea of what I can and can't do with NodeJS, about CI/CD, RESTful APIs, file format validations, QRCode generation, or even static site generators and JSON.What is it ?Website: https://ift.tt/2siwjys repository: https://ift.tt/2JcGcba https://ift.tt/2sidWcQ sum up what the app does : it's a karaoke session manager. You're with some friends or with a crowd, you start up the app, it starts up mpv (http://mpv.io) and a webserver using Express to allow your guests to add suggestions to the playlist or search for songs to add. It uses SQLite3 for its backend database (since it's a standalone app) and we started writing the admin backoffice in React (the main frontend is handcrafted JS with a few libraries thrown in). The guests use their own devices to access the server (they must be on the same wifi network) and the admin can control the player and playlists from a specific admin panel.It uses a karaoke database another team of our own maintain (https://ift.tt/2kwc78R) but people can use their own karaoke files (as long as they use the same format we decided on)What was interesting during this first year of development was getting back on track with most modern ways of coding and working on a project. For the background I'm mostly a sysadmin with some dev knowledge, but over the years, I lost touch with what was happening in the dev world. NodeJS and this project allowed me to catch up.So why NodeJS ?We had a few constraints : the app had to be standalone and multi-platform. It needed to be as simple as possible and self-contained so users only had to double-click on an executable to launch the app.We achieved the standalone part thanks to pkg (https://ift.tt/2pJcVLV) and found out NodeJS had a pretty extensive ecosystem of modules we could use, either for database or for parsing ASS files (subtitles) or manipulating mpv. Using those was a breeze and we had a prototype running in a few weeks.From there, we started working on the web interface and adding other quality of life features.The code was rewritten between 2.0 and 2.1 from using nested promises with lots of then/catch to using async/await, which greatly simplified everything. We also started writing less monolithic code and spread functions more accross smaller JS files. It took us about 1 month to rewrite everything using more recent ECMAScript standards.In retrospect, what was really nice with NodeJS was that we coudl get something working very quickly and with good enough results for a prototype. Development started in early June, and our first real public release, 2.0, landed in early novemberWhy SQLite3A note on databases : I heard a lot of questions on why we used SQLite instead of anything else. SQLite is, as its name says "lite" and we were a bit frustrated with some performance issues when 20+ users tried to add songs or update the database at once. Clearly SQLite is not made to handle many concurrent users, but with some optimizations left and right, we managed to mitigate most of the slowdowns. Someone is working on a benchmark using Gatling and so far, the app starts sweating heavily at around 150 users simultaneously, which would be a very rare case (getting 150 people in a room to sing karaoke is pretty difficult :p)However, we couldn't afford to use another database like postgres or mongo. We needed to embed the database and its engine within the app. This was the simplest way to achieve that.Dependencies and the node_modules bloatI guess everyone is facing sooner or later a time when they look at the node_modules directory size and gasp in horror.When bundling your app like we do with pkg, remember that you can use --production with Yarn to only install production dependencies only and not the dev ones like eslint or babel, which tend to take a lot of space.Something else I did was looking at each module's size and looked at our yarn.lock to quickly check which dependency used it. By only takign intoa ccount modules bigger than 300-400Kb I quickly eliminated stuff I didn't need. One good example was express-validator, or lodash which take a lot of space. I only used a handful of functions from lodash and thus only added those as dependencies, not the whole package. Getting rid of momentJS also allowed to save space : I rewrote by hand the only function we used from it.Working with some module authors also proved to be helpful. I notified the node-mpv (https://ift.tt/2kuKk8A) author that it was using an older version of cuid, which had a big (2Mb) memory footprint. Upgrading it saved lots more space.Last but not least, I used modclean (https://ift.tt/2GXmmeY) with its safe setting to get rid of all non important files in the node_module directory prior to packaging.All in all, I saved up like 40 Mb of space (the .exe file went from 150 to 100-110 Mb in size)What's next ?We still have a lot of work on the app. It's considered stable and when you take a look at the issues left there aren't many new features we'd need to implement.However, we received some requests, notably to have stats uploaded to a central server, to know which songs are most popular, etc. There is also an issue about making the app work online, without any wifi network. If you've played games like Jackbox Party Pack before you'll have an idea of what I'm talking about : the app connects to an online server which proxifies requests from users (connected to the online server) back to the app running to control its player.We'd also like to rewrite the user frontend part in React too.These are complex issues and we're still trying to decide on how to tackle them.ConclusionNodeJS has been a fun experience for this little project, and it's not over yet. If you have questions about the app, about some choices we made, I'll try to answer all questions :) I probably didn't think of everything when writing this post.If you want to contribute, that'd be awesome! As I said above with have some challenges awaiting us, or we could need help refactoring stuff or cleaning up the code. Any help is welcome.
Submitted May 28, 2018 at 10:53AM by AxelTerizaki
No comments:
Post a Comment