ZK World's multi-dimensional parallel system
ZK World uses high-dimensional tensor parallelism to improve efficiency. The principle behind this is mainly to design tensor parallelism in a two-dimensional way. Tensor parallelism allows computing tasks to be decomposed and performed synchronously. The way of two-dimensional slicing allows each machine to only need to deal with machines in the same row or in the same column, and does not need to deal with all machines. "If we want 10,000 machines to calculate, in the traditional method (one-dimensional), one machine needs to deal with 9999 remaining machines, and we only need to deal with 99 machines."
Last updated