Speeding up computation by providing programmable interfaces to NSWAP

Speeding up computation by providing programmable interfaces to NSWAP

Colin Schimmelfing and Tia Newhall

Running scientific programs on computers, even high powered ones, can take a long time. Already NSWAP (Network Swapping) can help these programs by seamlessly providing faster swap space: extra RAM for a system. Normally, when a computer runs out of RAM it will use a hard disk to provide more. This extra RAM, however, is about one million times slower than regular RAM, since data accesses on disks are so slow. Instead NSWAP uses idle RAM of other machines on the network to provide this extra RAM, speeding up execution of programs. While this is good for most situations, many programs would be better served by NSWAP in other ways that are not currently available. For instance, it might be useful for a programmer to determine which parts of a program remain on local RAM and which ones reside on the network. Alternatively, for data-intensive workloads, particularly ones in which data is rewritten repeatedly, it might be beneficial to store the data on a filesystem on the network, instead of on disk. This case was addressed this summer, and NSWAP now supports filesystems just as any disk would, but with quicker access to files. Running a mock up of the I/O for the widely-used external merge sort algorithm, NSWAP reduced the I/O time by about 40% as a filesystem. While this is impressive, there is much more to be done in giving greater control to programmers so that NSWAP can be used most effectively for any need.