Prev – The “eureka” moment – discovery of RAM Disk Drives
Now there was no stopping, I did not need an SSD, all I had to do was install the RAM drive on our actual Jenkins windows machine, point SAS temporary data set creation to tmpfs and get rid of all the database IO issues and bring the build time on the Jenkins machine from 1+ hour to 25/30 minutes. Ajay allotted more RAM to both the Jenkins and the SAS Linux VMs. I did the needful and triggered the build process and to my astonishment, the build timings did not reduce much on our actual Jenkins and SAS Linux machines. I thought I did not do something right. I reconfirmed the changes on both the machines, re triggered the build and the result was same. I was completely shocked and disappointed. Though it was supposed to work, my hypothesis had failed and I was wondering why. The only reason I could imagine for the hypothesis to not work was that probably the SAS data set operations and the MySQL database operations that we were performing via tests were not as IO intensive as I had assumed earlier and there was something else in play. This was my new hypothesis.
Next – Test on smaller data set
Prev – The discovery of Ant JUnit task options
We use MySQL and most of our tests make use of database calls. We use empty db with seed data for our tests and the tests themselves create the required test data, invoke the method under test and then assert the output in db. Involvement of lot of database IO is one of the reason why our tests were running slow, at least that’s what my assumption was. Now I am not in a position to reduce the db dependency in our tests immediately, I still had to get around the problem immediately. We run SAS on Linux VM and my second assumption was that the SAS dataset operations are IO intensive. So now it was clear in my mind that SSD is going to solve the rest of my problem.
My colleague Ajay from IT, helped me by setting up a VM Server with hybrid disk and one physical desktop machine with SSD for me to test the Jenkins timings. After setting it up, both the machines brought down the timing of the jobs significantly. From 1+ hour we were down to about 35 minutes. The physical desktop with SSD was faster than the VM Server with hybrid drive. Now procuring VM servers with SDD was expensive and time consuming, requiring budget changes and approvals. Hybrid drives are cheaper than SSDs. In the meantime Ajay started working on hardware to figure out why VM with hybrid drive was not performing to our expectations. I am hoping Ajay will publish his own blog on his experience.
Next – The alternative for SSD, in-memory/in-process db