Distributed Programming

5 Key excerpts on "Distributed Programming"

• 

  eBook - ePub

  Computer Architecture and Security

  Fundamentals of Designing Secure Computer Systems

  • Shuangbao Paul Wang, Robert S. Ledley(Authors)
  • 2012(Publication Date)
  • Wiley

    (Publisher)

  Distributed computing is a field of computer science that studies distributed systems. A distributed system consists of multiple autonomous computers that communicate through a computer network. The computers interact with each other in order to achieve a common goal. A computer program that runs in a distributed system is called a distributed program, and Distributed Programming is the process of writing such programs.

  Distributed computing also refers to the use of distributed systems to solve computational problems. In distributed computing, a problem is divided into many tasks, each of which is solved by one or more computers.

  The word distributed in terms such as “distributed system,” “Distributed Programming,” and “distributed algorithm” originally referred to computer networks where individual computers were physically distributed within some geographical area. The terms are nowadays used in a much wider sense, even referring to autonomous processes that run on the same physical computer and interact with each other by message passing.

  7.4.4 Distributed Systems

  While there is no single definition of a distributed system, the following defining properties are commonly used:

  • There are several autonomous computational entities, each of which has its own local memory.
  • The entities communicate with each other by message passing.
  • Have a common goal, such as solving a large computational problem.
  • Each computer may have its own user with individual needs, and the purpose of the distributed system is to coordinate the use of shared resources or provide communication services to the users.
  • The system has to tolerate failures in individual computers.
  • The structure of the system (network topology, network latency, number of computers) is not known in advance, the system may consist of different kinds of computers and network links, and the system may change during the execution of a distributed program.

  7.4.5 Parallel and Distributed Computing

  Distributed systems are groups of networked computers, which have the same goal for their work. The terms “concurrent computing,” “parallel computing,” and “distributed computing” have a lot of overlap, and no clear distinction exists between them. The same system may be characterized both as “parallel” and “distributed;” the processors in a typical distributed system run concurrently in parallel. Parallel computing may be seen as a particular tightly-coupled form of distributed computing, and distributed computing may be seen as a loosely-coupled form of parallel computing. Nevertheless, it is possible to roughly classify concurrent systems as “parallel” or “distributed” using the following criteria:

  Sign up to read

  Learn more about book
• 

  eBook - ePub

  Distributed Computing with Python

  • Francesco Pierfederici(Author)
  • 2016(Publication Date)
  • Packt Publishing

    (Publisher)

  Distributing computation across multiple computers is again a pretty obvious strategy when using systems that are able to speak to each other over the (local or otherwise) network. In many respects, in fact, this is just a generalization of the concepts of parallel computing that we saw in the previous section.

  Reasons to build distributed systems abound. Oftentimes, the reason is the ability to tackle a problem so big that no individual computer could handle it at all, or at least, not in a reasonable amount of time. An interesting example from a field that is probably familiar to most of us is the rendering of 3D animation movies, such as those from Pixar and DreamWorks.

  Given the sheer number of frames to render for a full-length feature (30 frames per second on a two-hour movie is a lot!), movie studios need to spread the full-rendering job to large numbers of computers (computer farms as they are called).

  Other times, the very nature of the application being developed requires a distributed system. This is the case, for instance, for instant messaging and video conferencing applications. For these pieces of software, performance is not the main driver. It is just that the problem that the application solves is itself distributed.

  In the following figure, we see a very common web application architecture (another example of a distributed application), where multiple users connect to the website over the network. At the same time, the application itself communicates with systems (such as a database server) running on different machines in its LAN:

  Another interesting example of distributed systems, which might be a bit counterintuitive, is the CPU-GPU combination. These days, graphics cards are very sophisticated computers in their own right. They are highly parallel and offer compelling performance for a large number of compute-intensive problems, not just for displaying images on screen. Tools and libraries exist to allow programmers to make use of GPUs for general-purpose computing (for example CUDA from NVIDIA, OpenCL , and OpenAcc

  Sign up to read

  Learn more about book
• 

  eBook - ePub

  Parallel Computing Architectures and APIs

  IoT Big Data Stream Processing

  • Vivek Kale(Author)
  • 2019(Publication Date)
  • Chapman and Hall/CRC

    (Publisher)
• • Loosely coupled multicomputers (without shared memory) physically colocated. These may be bus-based (e.g., networks of workstations (NOW) connected by a local area network [LAN] or Myrinet card) or use a more general communication network, and the processors may be heterogeneous. In such systems, processors neither share memory nor have a common clock, and hence may be classified as distributed systems; however, the processors are very close to one another, which is characteristic of a parallel system. As the communication latency may be significantly lower than in wide area distributed systems, the solution approaches to various problems may be different for such systems than for wide area distributed systems.
• • Loosely coupled multicomputers (without shared memory and without a common clock) that are physically remote. These correspond to the conventional notion of distributed systems.
• • Parallelism or speedup of a program on a specific system
• This is a measure of the relative speedup of a specific program on a given machine. The speedup depends on the number of processors and the mapping of the code to the processors. It is expressed as the ratio of time T (1) with a single processor, to time T (n ) with n processors.
• • Parallelism within a parallel/distributed program
• This is an aggregate measure of the percentage of time that all the processors are executing CPU instructions productively, as opposed to waiting for communication (either via shared memory or message passing) operations to complete. The term is traditionally used to characterize parallel programs. If the aggregate measure is a function of only the code, then the parallelism is independent of the architecture.
• • Concurrency of a program
• This is similar to the parallelism of a program, but is used in the context of distributed programs. The parallelism/concurrency in a parallel/distributed program can be measured by the ratio of the number of local (non-communication and non-shared memory access) operations to the total number of operations (including the communication or shared memory access operations).
• • Granularity of a program
• The ratio of the amount of computation to the amount of communication within the parallel/distributed program is termed granularity