Prev – The “eureka” moment – discovery of RAM Disk Drives
Now there was no stopping, I did not need an SSD, all I had to do was install the RAM drive on our actual Jenkins windows machine, point SAS temporary data set creation to tmpfs and get rid of all the database IO issues and bring the build time on the Jenkins machine from 1+ hour to 25/30 minutes. Ajay allotted more RAM to both the Jenkins and the SAS Linux VMs. I did the needful and triggered the build process and to my astonishment, the build timings did not reduce much on our actual Jenkins and SAS Linux machines. I thought I did not do something right. I reconfirmed the changes on both the machines, re triggered the build and the result was same. I was completely shocked and disappointed. Though it was supposed to work, my hypothesis had failed and I was wondering why. The only reason I could imagine for the hypothesis to not work was that probably the SAS data set operations and the MySQL database operations that we were performing via tests were not as IO intensive as I had assumed earlier and there was something else in play. This was my new hypothesis.
Next – Test on smaller data set
Prev – The alternative for SSD, in-memory/in-process db
I was feeling very impatient and was wondering what to do. Was there an alternative to SSD? While thinking over it and with the word “memsql”, I had an “eureka” moment. I was wondering what if I was able to create a drive in RAM and have the entire MySQL db on it. Now, though it won’t be in-process, it would be completely in-memory and without any code changes it would be faster than any hard disk, hybrid disk, SSD. That was just a hypothesis and to my surprise I found that there are software’s that would convert extra RAM into RAM drive. I also came to know that Linux (any Unix flavor) already has this built-in with its tmpfs. I also came across these two blogs “RAMDisks Roundup and testing”, “RAM Disk Software Benchmarked”. Based on available benchmarks I shortlisted “SoftPerfect RAM Disk”. I was super excited and I wanted to immediately try it on the test VM servers and the build time did came down further!!!. The hypothesis had worked, the build time on the test VM Server came down to 25 minutes.
Next – The excitement and the disappointment