Tensor parallelism

Serving technique that splits each matrix (tensor) of a model across multiple GPUs: each card computes a part of the same layer and the results are recomposed. Used when a single layer doesn't fit on one GPU, e.g. for large MoE models.