|
337 | 337 |
|
338 | 338 | ## AI.MODELRUN |
339 | 339 |
|
340 | | -_This command is deprecated and will not be available in future versions. consider using AI.MODELEXECUTE command instead._ |
| 340 | +_This command is deprecated and will not be available in future versions. consider using `AI.MODELEXECUTE` command instead._ |
341 | 341 |
|
342 | 342 | The **`AI.MODELRUN`** command runs a model stored as a key's value using its specified backend and device. It accepts one or more input tensors and store output tensors. |
343 | 343 |
|
@@ -730,7 +730,110 @@ redis> > AI._SCRIPTSCAN |
730 | 730 | 2) "myscript:v0.1" |
731 | 731 | ``` |
732 | 732 |
|
| 733 | +## AI.DAGEXECUTE |
| 734 | +The **`AI.DAGEXECUTE`** command specifies a direct acyclic graph of operations to run within RedisAI. |
| 735 | + |
| 736 | +It accepts one or more operations, split by the pipe-forward operator (`|>`). |
| 737 | + |
| 738 | +By default, the DAG execution context is local, meaning that tensor keys appearing in the DAG only live in the scope of the command. That is, setting a tensor with `TENSORSET` will store it local memory and not set it to an actual database key. One can refer to that key in subsequent commands within the DAG, but that key won't be visible outside the DAG or to other clients - no keys are open at the database level. |
| 739 | + |
| 740 | +Loading and persisting tensors from/to keyspace should be done explicitly. The user should specify which key tensors to load from keyspace using the `LOAD` keyword, and which command outputs to persist to the keyspace using the `PERSIST` keyspace. The user can also specify keys in Redis that are going to be accessed for read/write operations (for example, from within `AI.SCRIPTEXECUTE` command), by using the keyword `KEYS`. |
| 741 | + |
| 742 | +As an example, if `command 1` sets a tensor, it can be referenced by any further command on the chaining. |
| 743 | + |
| 744 | +A `TIMEOUT t` argument can be specified to cause a request to be removed from the queue after it sits there `t` milliseconds, meaning that the client won't be interested in the result being computed after that time (`TIMEDOUT` is returned in that case). Note that individual `MODELEXECUTE` or `SCRIPTEXECUTE` commands within the DAG do not support `TIMEOUT`. `TIMEOUT` only applies to the `DAGEXECUTE` request as a whole. |
| 745 | + |
| 746 | + |
| 747 | +**Redis API** |
| 748 | + |
| 749 | +``` |
| 750 | +AI.DAGEXECUTE [[LOAD <n> <key-1> <key-2> ... <key-n>] | |
| 751 | + [PERSIST <n> <key-1> <key-2> ... <key-n>] | |
| 752 | + [KEYS <n> <key-1> <key-2> ... <key-n>]]+ |
| 753 | + [TIMEOUT t] |
| 754 | + |> <command> [|> command ...] |
| 755 | +``` |
| 756 | + |
| 757 | +_Arguments_ |
| 758 | + |
| 759 | +* **LOAD**: denotes the beginning of the input tensors keys' list, followed by the number of keys, and one or more key names |
| 760 | +* **PERSIST**: denotes the beginning of the output tensors keys' list, followed by the number of keys, and one or more key names |
| 761 | +* **KEYS**: denotes the beginning of keys' list which are used within this command, followed by the number of keys, and one or more key names. Alternately, the keys names list can be replaced with a tag which all of those keys share. Redis will verify that all potential key accesses are done to the right shard. |
| 762 | + |
| 763 | +_While each of the LOAD, PERSIST and KEYS sections are optional (and may appear at most once in the command), the command must contain **at least one** of these 3 keywords._ |
| 764 | +* **TIMEOUT**: an optional argument, denotes the time (in ms) after which the client is unblocked and a `TIMEDOUT` string is returned |
| 765 | +* **|> command**: the chaining operator, that denotes the beginning of a RedisAI command, followed by one of RedisAI's commands. Command splitting is done by the presence of another `|>`. The supported commands are: |
| 766 | + * `AI.TENSORSET` |
| 767 | + * `AI.TENSORGET` |
| 768 | + * `AI.MODELEXECUTE` |
| 769 | + * `AI.SCRIPTEXECUTE` |
| 770 | + |
| 771 | + |
| 772 | +`AI.MODELEXECUTE` and `AI.SCRIPTEXECUTE` commands can run on models or scripts that were set on different devices. RedisAI will analyze the DAG and execute commands in parallel if they are located on different devices and their inputs are available. |
| 773 | +Note that KEYS should not be specified in `AI.SCRIPTEXECUTE` commands of the DAG. |
| 774 | + |
| 775 | +_Return_ |
| 776 | + |
| 777 | +An array with an entry per command's reply. Each entry format respects the specified command reply. |
| 778 | +In case the `DAGEXEUTE` request times out, a `TIMEDOUT` simple string is returned. |
| 779 | + |
| 780 | +**Examples** |
| 781 | + |
| 782 | +Assuming that running the model that's stored at 'mymodel', we define a temporary tensor 'mytensor' and use it as input, and persist only one of the two outputs - discarding 'classes' and persisting 'predictions'. In the same command return the tensor value of 'predictions'. The following command does that: |
| 783 | + |
| 784 | + |
| 785 | +``` |
| 786 | +redis> AI.DAGEXECUTE PERSIST 1 predictions{tag} |> |
| 787 | + AI.TENSORSET mytensor FLOAT 1 2 VALUES 5 10 |> |
| 788 | + AI.MODELEXECUTE mymodel{tag} INPUTS 1 mytensor OUTPUTS 2 classes predictions{tag} |> |
| 789 | + AI.TENSORGET predictions{tag} VALUES |
| 790 | +1) OK |
| 791 | +2) OK |
| 792 | +3) 1) FLOAT |
| 793 | + 2) 1) (integer) 2 |
| 794 | + 2) (integer) 2 |
| 795 | + 3) "\x00\x00\x80?\x00\x00\x00@\x00\x00@@\x00\x00\x80@" |
| 796 | +``` |
| 797 | + |
| 798 | +A common pattern is enqueuing multiple SCRIPTEXECUTE and MODELEXECUTE commands within a DAG. The following example uses ResNet-50,to classify images into 1000 object categories. Given that our input tensor contains each color represented as a 8-bit integer and that neural networks usually work with floating-point tensors as their input we need to cast a tensor to floating-point and normalize the values of the pixels - for that we will use `pre_process_3ch` function. |
| 799 | + |
| 800 | +To optimize the classification process we can use a post process script to return only the category position with the maximum classification - for that we will use `post_process` script. Using the DAG capabilities we've removed the necessity of storing the intermediate tensors in the keyspace. You can even run the entire process without storing the output tensor, as follows: |
| 801 | + |
| 802 | +``` |
| 803 | +redis> AI.DAGEXECUTE KEYS 1 {tag} |> |
| 804 | + AI.TENSORSET image UINT8 224 224 3 BLOB b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00....' |> |
| 805 | + AI.SCRIPTEXECUTE imagenet_script{tag} pre_process_3ch INPUTS 1 image OUTPUTS 1 temp_key1 |> |
| 806 | + AI.MODELEXECUTE imagenet_model{tag} INPUTS 1 temp_key1 OUTPUTS 1 temp_key2 |> |
| 807 | + AI.SCRIPTEXECUTE imagenet_script{tag} post_process INPUTS 1 temp_key2 OUTPUTS 1 output |> |
| 808 | + AI.TENSORGET output VALUES |
| 809 | +1) OK |
| 810 | +2) OK |
| 811 | +3) OK |
| 812 | +4) OK |
| 813 | +5) 1) 1) (integer) 111 |
| 814 | +``` |
| 815 | + |
| 816 | +As visible on the array reply, the label position with higher classification was 111. |
| 817 | + |
| 818 | +By combining DAG with multiple SCRIPTEXECUTE and MODELEXECUTE commands we've substantially removed the overall required bandwith and network RX ( we're now returning a tensor with 1000 times less elements per classification ). |
| 819 | + |
| 820 | + |
| 821 | + |
| 822 | +!!! warning "Intermediate memory overhead" |
| 823 | + The execution of models and scripts within the DAG may generate intermediate tensors that are not allocated by the Redis allocator, but by whatever allocator is used in the backends (which may act on main memory or GPU memory, depending on the device), thus not being limited by `maxmemory` configuration settings of Redis. |
| 824 | + |
| 825 | +## AI.DAGEXECUTE_RO |
| 826 | + |
| 827 | +The **`AI.DAGEXEUTE_RO`** command is a read-only variant of `AI.DAGEXECUTE`. |
| 828 | +`AI.DAGEXECUTE` is flagged as a 'write' command in the Redis command table (as it provides the `PERSIST` option, for example). Hence, read-only cluster replicas will refuse to run the command and it will be redirected to the master even if the connection is using read-only mode. |
| 829 | + |
| 830 | +`AI.DAGEXECUTE_RO` behaves exactly like the original command, excluding the `PERSIST` option and `AI.SCRIPTEXECUTE` commands. It is a read-only command that can safely be with read-only replicas. |
| 831 | + |
| 832 | +!!! info "Further reference" |
| 833 | + Refer to the Redis [`READONLY` command](https://redis.io/commands/readonly) for further information about read-only cluster replicas. |
| 834 | + |
733 | 835 | ## AI.DAGRUN |
| 836 | +_This command is deprecated and will not be available in future versions. consider using `AI.DAGEXECUTE` command instead._ |
734 | 837 | The **`AI.DAGRUN`** command specifies a direct acyclic graph of operations to run within RedisAI. |
735 | 838 |
|
736 | 839 | It accepts one or more operations, split by the pipe-forward operator (`|>`). |
@@ -817,7 +920,7 @@ By combining DAG with multiple SCRIPTRUN and MODELRUN commands we've substantial |
817 | 920 | The execution of models and scripts within the DAG may generate intermediate tensors that are not allocated by the Redis allocator, but by whatever allocator is used in the backends (which may act on main memory or GPU memory, depending on the device), thus not being limited by `maxmemory` configuration settings of Redis. |
818 | 921 |
|
819 | 922 | ## AI.DAGRUN_RO |
820 | | - |
| 923 | +_This command is deprecated and will not be available in future versions. consider using `AI.DAGEXECUTE_RO` command instead._ |
821 | 924 | The **`AI.DAGRUN_RO`** command is a read-only variant of `AI.DAGRUN`. |
822 | 925 |
|
823 | 926 | Because `AI.DAGRUN` provides the `PERSIST` option it is flagged as a 'write' command in the Redis command table. However, even when `PERSIST` isn't used, read-only cluster replicas will refuse to run the command and it will be redirected to the master even if the connection is using read-only mode. |
|
0 commit comments