This chapter is from the book
This chapter is from the book
This chapter is from the book
Grid Infrastructure
The grid infrastructure forms the core foundation for successful grid applications. This infrastructure is a complex combination of a number of capabilities and resources identified for the specific problem and environment being addressed.
In initial stages of delivering any Grid Computing application infrastructure, the developers/service providers must consider the following questions in order to identify the core infrastructure support required for that environment:
-
What problem(s) are we trying to solve for the user? How do we address grid enablement simpler, while addressing the user's application simpler? How does the developer (programmatically) help the user to be able to quickly gain access and utilize the application to best fit their problem resolution needs?
-
How difficult is it to use the grid tool? Are grid developers providing a flexible environment for the intended user community?
-
Is there anything not yet considered that would make it easier for grid service providers to create tools for the grid, suitable for the problem domain?
-
What are the open standards, environments, and regulations grid service providers must address?
In the early development stages of grid applications, numerous vertical "towers" and middleware solutions were often developed to solve Grid Computing problems. These various middleware and solution approaches were developed for fairly narrow and limited problem-solving domains, such as middleware to deal with numerical analysis, customized data access grids, and other narrow problems. Today, with the emergence and convergence of grid service-oriented technologies,[3] including the interoperable XML[4]-based solutions becoming ever more present and industry providers with a number of reusable grid middleware solutions facilitating the following requirement areas, it is becoming simpler to quickly deploy valuable solutions. Figure 1.4 shows this topology of middleware topics.
)
Figure 1.4. Grid middleware topic areas are becoming more sophisticated at an aggressive rate.
In general, a Grid Computing infrastructure component must address several potentially complicated areas in many stages of the implementation. These areas are:
-
Security
-
Resource management
-
Information services
-
Data management
Let us further examine the significance of each of these above components.
Security
The heterogeneous nature of resources and their differing security policies are complicated and complex in the security schemes of a Grid Computing environment. These computing resources are hosted in differing security domains and heterogeneous platforms. Simply speaking, our middleware solutions must address local security integration, secure identity mapping, secure access/authentication, secure federation, and trust management.
The other security requirements are often centered on the topics of data integrity, confidentiality, and information privacy. The Grid Computing data exchange must be protected using secure communication channels, including SSL/TLS and oftentimes in combination with secure message exchange mechanisms such as WS-Security. The most notable security infrastructure used for securing grid is the Grid Security Infrastructure (GSI). In most cases, GSI provides capabilities for single sign-on, heterogeneous platform integration and secure resource access/authentication.
The latest and most notable security solution is the use of WS-Security standards. This mechanism provides message-level, end-to-end security needed for complex and interoperable secure solutions. In the coming years we will see a number of secure grid environments using a combination of GSI and WS-Security mechanisms for secure message exchanges. We will discuss the details of security mechanisms provided by these standards later in this book.
Resource Management
The tremendously large number and the heterogeneous potential of Grid Computing resources causes the resource management challenge to be a significant effort topic in Grid Computing environments. These resource management scenarios often include resource discovery, resource inventories, fault isolation, resource provisioning, resource monitoring, a variety of autonomic capabilities,[5] and service-level management activities. The most interesting aspect of the resource management area is the selection of the correct resource from the grid resource pool, based on the service-level requirements, and then to efficiently provision them to facilitate user needs.
Let us explore an example of a job management system, where the resource management feature identifies the job, allocates the suitable resources for the execution of the job, partitions the job if necessary, and provides feedback to the user on job status. This job scheduling process includes moving the data needed for various computations to the appropriate Grid Computing resources, and mechanisms for dispatching the job results.
It is important to understand multiple service providers can host Grid Computing resources across many domains, such as security, management, networking services, and application functionalities. Operational and application resources may also be hosted on different hardware and software platforms. In addition to this complexity, Grid Computing middleware must provide efficient monitoring of resources to collect the required matrices on utilization, availability, and other information.
One causal impact of this fact is (as an example) the security and the ability for the grid service provider to reach out and probe into other service provider domains in order to obtain and reason about key operational information (i.e., to reach across a service provider environment to ascertain firewall and router volume-related specifics, or networking switch status, or application server status). This oftentimes becomes complicated across several dimensions, and has to be resolved by a meeting-of-the-minds between all service providers, such as messaging necessary information to all providers, when and where it is required.
Another valuable and very critical feature across the Grid Computing infrastructure is found in the area of provisioning; that is, to provide autonomic capabilities for self-management, self-diagnosis, self-healing, and self-configuring. The most notable resource management middleware solution is the Grid Resource Allocation Manager (GRAM). This resource provides a robust job management service for users, which includes job allocation, status management, data distribution, and start/stop jobs.
Information Services
Information services are fundamentally concentrated on providing valuable information respective to the Grid Computing infrastructure resources. These services leverage and entirely depend on the providers of information such as resource availability, capacity, and utilization, just to name a few. This information is valuable and mandatory feedback respective to the resources managers discussed earlier in this chapter. These information services enable service providers to most efficiently allocate resources for the variety of very specific tasks related to the Grid Computing infrastructure solution.
In addition, developers and providers can also construct grid solutions to reflect portals, and utilize meta-schedulers and meta-resource managers. These metrics are helpful in service-level management (SLA) in conjunction with the resource policies. This information is resource specific and is provided based on the schema pertaining to that resource. We may need higher level indexing services or data aggregators and transformers to convert these resource-specific data into valuable information sources for the end user.
For example, a resource may provide operating system information, while yet another resource might provide information on hardware configuration, and we can then group this resource information, reason with it, and then suggest a "best" price combination on selecting the operating system on other certain hardware. This combinatorial approach to reasoning is very straightforward in a Grid Computing infrastructure, simply due to the fact that all key resources are shared, as is the information correlated respective to the resources.
Data Management
Data forms the single most important asset in a Grid Computing system. This data may be input into the resource, and the results from the resource on the execution of a specific task. If the infrastructure is not designed properly, the data movement in a geographically distributed system can quickly cause scalability problems. It is well understood that the data must be near to the computation where it is used. This data movement in any Grid Computing environment requires absolutely secure data transfers, both to and from the respective resources. The current advances surrounding data management are tightly focusing on virtualized data storage mechanisms, such as storage area networks (SAN), network file systems, dedicated storage servers, and virtual databases. These virtualization mechanisms in data storage solutions and common access mechanisms (e.g., relational SQLs, Web services, etc.) help developers and providers to design data management concepts into the Grid Computing infrastructure with much more flexibility than traditional approaches.
Some of the considerations developers and providers must factor into decisions are related to selecting the most appropriate data management mechanism for Grid Computing infrastructures. This includes the size of the data repositories, resource geographical distribution, security requirements, schemes for replication and caching facilities, and the underlying technologies utilized for storage and data access.
So far in this introductory chapter we have been discussing the details surrounding many aspects of the middleware framework requirements, specifically the emergence of service provider-oriented architectures[6] and, hence, the open and extremely powerful utility value of XML-based interoperable messages. These combined, provide a wide range of capabilities that deal with interoperability problems, and come up with a solution that is suitable for the dynamic virtual organizational grids. The most important activity noted today in this area is the Open Grid Service Architecture (OGSA) and its surrounding standard initiatives. Significant detail is recorded on this architecture, and will be given full treatment in subsequent chapters in this book. The OGSA provides a common interface solution to grid services, and all the information has been conveniently encoded using XML as the standard. This provides a common approach to information services and resource management for Grid Computing infrastructures.
This introductory chapter has discussed many of the chapters and some of their detail that will be presented throughout this book. This introductory discussion has been presented at a high level, and more detailed discussions with simple-to-understand graphics will follow.