Parallel vs Distributed Algorithms

Let me clear first that I am not asking about parallel (data/task) or distributed computing architecture. A lot had been discussed here. I am just asking a plain theoretical question. What is core principal difference between Parallel and Distributed Algorithms? Below are my under standings:

In parallel algorithms (task parallelism), A big task is divided into two or more sub tasks and each sub task is executed by one processing element (PE) parallely. All sub tasks are depended on each other. What I further understand, a parallel algorithm can be coded using OpenMP which runs on multiprocessors as well as can be coded using MPI which runs on message passing systems. on the other hand, in distributed algorithm, every processing element or node executes the same copy of algorithm. They are totally independent and exchange messages to achieve the common goal. I think distributed algorithms are designed only for multi-computer systems.

It is requested to provide parallel and distributed algorithm for a same example problem for better understanding.

An algorithm is parallel if there are several processes (tasks, threads, processors) working on it at the same time. Often the tasks run in the same address space, and can communicate/reference results by others freely (low cost).

An algorithm is distributed if it is parallel and the tasks run on separate machines (separate address spaces), one task has no direct access to the work of the others. It has to request needed data, or just wait until it is sent to it.

Yes, it is a fuzzy distinction.

StackExchange Computer Science Q#51099, answer score: 11