We use MySQL and most of our tests make use of database calls. We use empty db with seed data for our tests and the tests themselves create the required test data, invoke the method under test and then assert the output in db. Involvement of lot of database IO is one of the reason why our tests were running slow, at least that’s what my assumption was. Now I am not in a position to reduce the db dependency in our tests immediately, I still had to get around the problem immediately. We run SAS on Linux VM and my second assumption was that the SAS dataset operations are IO intensive. So now it was clear in my mind that SSD is going to solve the rest of my problem.
My colleague Ajay from IT, helped me by setting up a VM Server with hybrid disk and one physical desktop machine with SSD for me to test the Jenkins timings. After setting it up, both the machines brought down the timing of the jobs significantly. From 1+ hour we were down to about 35 minutes. The physical desktop with SSD was faster than the VM Server with hybrid drive. Now procuring VM servers with SDD was expensive and time consuming, requiring budget changes and approvals. Hybrid drives are cheaper than SSDs. In the meantime Ajay started working on hardware to figure out why VM with hybrid drive was not performing to our expectations. I am hoping Ajay will publish his own blog on his experience.