You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Jan 26, 2026. It is now read-only.
Instead of using mpirun to launch a set of ranks/processes, you can tell the runtime to
58
+
spawns ranks/processes for you by setting SHARPY_MPI_SPAWN to the number of desired MPI processes.
59
+
Additionally set SHARPY_MPI_EXECUTABLE and SHARPY_MPI_EXE_ARGS.
60
+
Additionally SHARPY_MPI_HOSTS can be used to control the host to use for spawning processes.
61
+
62
+
The following command will run the stencil example on 3 MPI ranks:
63
+
```bash
64
+
SHARPY_IDTR_SO=`pwd`/sharpy/libidtr.so \
65
+
SHARPY_MPI_SPAWN=2 \
66
+
SHARPY_MPI_EXECUTABLE=`which python` \
67
+
SHARPY_MPI_EXE_ARGS="examples/stencil-2d.py 10 2000 star 2" \
68
+
python examples/stencil-2d.py 10 2000 star 2
69
+
```
70
+
68
71
## Contributing
69
72
Please setup precommit hooks like this
70
73
```
@@ -78,11 +81,10 @@ Typically, sharpy operations do not get executed immediately. Instead, the funct
78
81
the actual computation gets deferred by creating a promise/deferred object and queuing it for later. This is not visible to users, they can use it as any other numpy-like library.
79
82
80
83
Only when actual data is needed, computation will happen; that is when
81
-
- the values of array elements are casted to bool intor float
82
-
- the array is printed
84
+
- the values of array elements are casted to bool, int, float or string
85
+
-this includes when the array is printed
83
86
84
-
In the background a worker thread handles deferred objects. Until computation is needed it dequeues deferred objects from the FIFO queue and asks them to generate MLIR.
85
-
Objects can either generate MLIR or instead provide a run() function to immediately execute. For the latter case the current MLIR function gets executed before calling run() to make sure potential dependences are met.
87
+
In the background a worker thread handles deferred objects. Until computation is needed it dequeues deferred objects from the FIFO queue and asks them to generate MLIR. Objects can either generate MLIR or instead provide a run() function to immediately execute. For the latter case the current MLIR function gets executed before calling run() to make sure potential dependencies are met.
86
88
87
89
### Distribution
88
90
Arrays and operations on them get transparently distributed across multiple processes. Respective functionality is partly handled by this library and partly IMEX dist dialect.
@@ -91,6 +93,25 @@ sharpy provides this library functionality in a separate dynamic library "idtr".
91
93
92
94
Right now, data is split in the first dimension (only). Each process knows the partition it owns. For optimization partitions can actually overlap.
93
95
94
-
sharpy supports to execution modes:
95
-
1. CSP/SPMD/explicitly-distributed execution, meaning all processes execute the same program, execution is replicated on all processes. Data is typically not replicated but distributed among processes.
96
-
2. Controller-Worker/implicitly-distributed execution, meaning only a single process executes the program and it distributes data and work to worker processes.
96
+
sharpy currently supports one execution mode: CSP/SPMD/explicitly-distributed execution, meaning all processes execute the same program, execution is replicated on all processes. Data is typically not replicated but distributed among processes. The distribution is handled automatically by sharpy, all operations on sharpy arrays can be viewed as collective operations.
97
+
98
+
Later, we'll add a Controller-Worker/implicitly-distributed execution mode, meaning only a single process executes the program and it distributes data and work to worker processes.
99
+
100
+
### Array API Coverage
101
+
Currently only a subset of the Array API is covered by sharpy
102
+
- elementwise binary operations
103
+
- elementwise unary operations
104
+
- subviews (getitem with slices)
105
+
- assignment (setitem with slices)
106
+
-`empty`, `zeros`, `ones`, `linspace`, `arange`
107
+
- reduction operations over all dimensions (max, min, sum, ...)
108
+
- type promotion
109
+
- many cases of shape broadcasting
110
+
111
+
### Other Functionality
112
+
-`sharpy.to_numpy` converts a sharpy array into a numpy array.
113
+
-`sharpy.numpy.from_function` allows creating a sharpy array from a function (similar to numpy)
114
+
- In addition to the Array API sharpy also provides functionality facilitating interacting with sharpy arrays in a distributed environment.
115
+
-`sharpy.spmd.gather` gathers the distributed array and forms a single, local and contiguous copy of the data as a numpy array
116
+
-`sharpy.spmd.get_locals` return the local part of the distributed array as a numpy array
117
+
- sharpy allows providing a fallback array implementation. By setting SHARPY_FALLBACK to a python package it will call that package if a given function is not provided by sharpy. It will pass sharpy arrays as (gathered) numpy-arrays.
0 commit comments