Widespread use of cluster systems in diverse set of applications has spurred significant interest in providing high performance cluster interconnects. A major inefficiency in utilizing such interconnects has been the send/receive communication overheads at the sender/receiver hosts. Various techniques such as User-Level Communication (ULC) have been proposed to mitigate this communication inefficiency. However, due to recent security breaches, focus on cluster communication security research has spurred. Such research is non-trivial due to the high-speed nature of the cluster interconnect. This paper surveys the four most popular cluster interconnects used in Top500 supercomputers and explores possible schemes to ensure secure cluster intra-communication encompassing the host processor, secure coprocessor and the Network Interface Card (NIC) by illustrating its challenges in doing so. We then compare these schemes in terms of host processor offload, end-to-end latency, security transparency and cryptographic processing performance. Then we give an overview of security issues for those cluster interconnects designs.
Conference & Workshops
I/O Node Placement for Performance and Reliability in Torus Networks
When a cluster system interconnects processor and I/O nodes through a network, an optimal placement of I/O nodes is critical to improve the overall system performance by reducing its communication latency. In this paper, we propose an efficient and scalable I/O node placement scheme, called a relaxed quasi-perfect, for torusbased interconnection networks using Lee distance errorcorrecting code. It provides a more general placement than the previous quasi-perfect placement [1]. We also suggest a fault-tolerant scheme using our I/O placement model for a guaranteed performance. Simulation results show that our scheme provides 53% speed-up over the quasi-perfect. Also the fault tolerant scheme provides a graceful slowdown especially until the number of faulty I/O nodes becomes less than half of the initial I/O nodes.
A Heuristic for Peak Power Constrained Design of Network on Chip (NoC) based Multimode System
Designing NoC-based systems has become increasingly complex with support for multiple functionalities. Decisions regarding interconnections between the heterogeneous system components and routing of system communication affect system performance and power consumption. This research provides a heuristic to determine the neighborhood configuration for each component. By controlling the communication bandwidth allocation, simulation results with synthetic and real workloads indicate that our heuristic is able to control the peak power consumption, but at cost of throughput degradation.
Security Enhancement in InfiniBand Architecture
The InfiniBandTM Architecture (IBA) is a new promising I/O communication standard positioned for building clusters and System Area Networks (SANs). However, the IBA specification has left out security resulting in potential security vulnerabilities, which could be exploited with moderate effort. In this paper, we view these vulnerabilities from three classical security aspects: availability, confidentiality, and authentication. For better availability of IBA, we recommend that a switch be able to enforce partitioning for data packets for which we propose an efficient implementation method using trap messages. For confidentiality, we encrypt only secret keys to minimize performance degradation. The most serious vulnerability in IBA is authentication since IBA authenticates packets solely by checking the existence of plaintext keys in the packet. In this paper, we propose a new authentication mechanism that treats the Invariant CRC (ICRC) field as an authentication tag, which is compatible with current IBA specification. When analyzing the performance of our authentication approach along with other authentication algorithms, we observe that our approach dramatically enhances IBA’s authentication capability without hampering IBA performance benefit. Furthermore, simulation results indicate that our methods enhance security in IBA with marginal performance overhead.
Application-Level Memory Management in Cluster-based Servers
Power Mangement in Disk Storage System with Multiple RPMs
Peak Power Control for a QoS Capable On-Chip Network
In recent years integrating multiprocessors in a single chip is emerging for supporting various scientific and commercial applications, with diverse demands to the underlying on-chip networks. Communication traffic of these applications makes routers greedy to acquire more power such that the total consumed power of the network may exceed the supplied power and cause reliability problems. To ensure high performance and power constraint satisfaction, the on-chip network must have a peak power control mechanism. In this paper, we propose a credit-based peak power control scheme to assure power consumption to be under the given peak power constraint, without performance degradation. The peak power control scheme efficiently regulates each flow’s injection rate at the sender to minimize performance penalty. We have two different throttling schemes for real-time traffic and best-effort traffic; a rate-based throttling and an energy-budget based throttling, respectively. The simulation results on mesh networks show that the credit-based peak power control effectively prevents performance degradation and meets the peak power constraint.
Instant Attack Stopper in InfiniBand Architecture
Proceedings of the Cluster-Sec 2005 Workshop (Cluster-Sec, CCGrid Workshops) 2005, Cardiff, UK, 2005
With the growing popularity of cluster architectures in datacenters and the sophistication of computer attacks, the design of highly secure clusters has recently emerged as a critical design issue. However, the majority of cluster security research has focused on how to detect and prevent attacks rather than on how to minimize the effect of attacks once detected. The action against detected attacks in the cluster is as important as the actual detection process since no detection mechanism is full-proof in its ability to protect cluster systems without the effective cluster-wide reaction. In this paper, we propose a scheme, referred to as the Instant Attack Stopper (IAS) that can instantly confront security attacks in a cluster. Specifically we provide detailed implementation methods of IAS in InfiniBand Architecture (IBA) – a new promising communication standard for future System Area Networks (SANs) and clusters. IAS focuses on removing malicious communication on the IBA fabric among processes involved in an attack, which is accomplished through the proposed Security Management Agent (SeMA). We will show IAS deployment in different security levels to meet various security requirements.
Power Management in RAID Server Disk System Using Multiple Idle States
Energy efficient design for disk devices has become a very important issue, because ever increasing need for data storage systems significantly increases the power budget in the disk array. Most research on power management in disk devices focuses on the traditional way of stopping disk rotation. In this paper, we propose a new approach in which the energy consumption can be reduced without stopping the disk rotation. The proposed Multiple Idle States (MIS) model modulates the disk RPM to optimize the energy consumption during idle periods. We present a new threshold time model to calculate a minimum idle period so as to save energy without significant performance degradation. The simulation results with real traces show that our MIS achieves more than 58% energy savings at low I/O workload and also gets 8% energy savings for high workload with less than 50% response time compared to the traditional shut-down power management scheme.
On Improving Performance and Conserving Power in Cluster-based Web Servers
With the growing use of cluster systems in web servers, file distribution and database transactions, power conservation and efficiency have been identified as critical issues in the design of cluster systems. Widely adopted, distributor-based systems forward client requests to a balanced set of backend servers in complete transparency to the clients. In this paper, we use power and locality-based request distribution at the distributor to provide optimum power conservation, while maintaining the required QoS of the system. The distribution scheme uses a simple memory management technique using pinned memory on the backend servers and proactive distribution, with the aid of data organization of the website, to improve the locality of the files. A simple on-off based power management scheme is applied to conserve power. Our scheme provides reduced response time to the clients and improved power conservation at the backend server cluster without compromising performance. Simulations involving real-time web traces and latest web technologies witness performance boost of 15- 23% and power conservation of 15-48% over the existing policies.