With the widespread use of cluster systems and ever increasing threat to computer security, it becomes more necessary to design and build secure cluster systems. Most cluster systems rely on security products like firewalls for their security, but they cannot guarantee security of intra-cluster communications, which can be a weak spot that hackers exploit for further security attacks. A recent study by Lee and Kim (2007) [22] proposed a security framework to protect intra-cluster communications by encrypting and authenticating all packets with fine-grained security where any two communicating processes dynamically generate and share a cryptographic key, called a session key. However, the fine-grained security scheme can incur serious performance degradation in large-scale cluster systems since it may take a long time to access session keys. To solve this problem, we propose to incorporate a session key cache inside a cluster interconnect card to speed up accesses to the session keys and build an analytical cluster traffic model to estimate the behavior of the cache in large-scale cluster systems. For further performance improvement, we propose a prefetching scheme speculating job scheduler’s decision without OS interventions. Simulation results indicate that the session key cache with the prefetching scheme decreases the network latency by 50% on average, compared to the configurations without the enhancements.