|
| 1 | +# FlowQuery Python Implementation |
| 2 | + |
| 3 | +This is the Python implementation of FlowQuery, a declarative query language for data processing pipelines. |
| 4 | + |
| 5 | +## Installation |
| 6 | + |
| 7 | +### From Source |
| 8 | + |
| 9 | +```bash |
| 10 | +git clone https://github.com/microsoft/FlowQuery.git |
| 11 | +cd FlowQuery/flowquery-py |
| 12 | +pip install -e . |
| 13 | +``` |
| 14 | + |
| 15 | +### With Development Dependencies |
| 16 | + |
| 17 | +```bash |
| 18 | +pip install -e ".[dev]" |
| 19 | +``` |
| 20 | + |
| 21 | +## Quick Start |
| 22 | + |
| 23 | +### Command Line Interface |
| 24 | + |
| 25 | +After installation, you can start the interactive REPL: |
| 26 | + |
| 27 | +```bash |
| 28 | +flowquery |
| 29 | +``` |
| 30 | + |
| 31 | +### Using Conda (Alternative) |
| 32 | + |
| 33 | +**Windows (PowerShell):** |
| 34 | + |
| 35 | +```powershell |
| 36 | +cd flowquery-py |
| 37 | +.\setup_env.ps1 |
| 38 | +conda activate flowquery |
| 39 | +``` |
| 40 | + |
| 41 | +**Linux/macOS:** |
| 42 | + |
| 43 | +```bash |
| 44 | +cd flowquery-py |
| 45 | +chmod +x setup_env.sh |
| 46 | +./setup_env.sh |
| 47 | +conda activate flowquery |
| 48 | +``` |
| 49 | + |
| 50 | +The setup scripts automatically: |
| 51 | + |
| 52 | +1. Read the Python version from `pyproject.toml` |
| 53 | +2. Create a conda environment named `flowquery` |
| 54 | +3. Install the package with all dev dependencies |
| 55 | + |
| 56 | +## Requirements |
| 57 | + |
| 58 | +- Python 3.10+ (defined in `pyproject.toml`) |
| 59 | +- pytest (for running tests) |
| 60 | +- pytest-asyncio (for async test support) |
| 61 | +- aiohttp (for HTTP requests) |
| 62 | + |
| 63 | +All dependencies are managed in `pyproject.toml`. |
| 64 | + |
| 65 | +## Programmatic Usage |
| 66 | + |
| 67 | +```python |
| 68 | +import asyncio |
| 69 | +from flowquery import Runner |
| 70 | + |
| 71 | +runner = Runner("WITH 1 as x RETURN x + 1 as result") |
| 72 | +asyncio.run(runner.run()) |
| 73 | +print(runner.results) # [{'result': 2}] |
| 74 | +``` |
| 75 | + |
| 76 | +## Running Tests |
| 77 | + |
| 78 | +```bash |
| 79 | +pytest tests/ |
| 80 | +``` |
| 81 | + |
| 82 | +## Project Structure |
| 83 | + |
| 84 | +``` |
| 85 | +flowquery-py/ |
| 86 | +├── pyproject.toml # Dependencies & project config (single source of truth) |
| 87 | +├── setup_env.ps1 # Windows conda setup script |
| 88 | +├── setup_env.sh # Linux/macOS conda setup script |
| 89 | +├── README.md |
| 90 | +├── src/ |
| 91 | +│ ├── __init__.py # Main package entry point |
| 92 | +│ ├── extensibility.py # Public API for custom functions |
| 93 | +│ ├── compute/ |
| 94 | +│ │ └── runner.py # Query execution engine |
| 95 | +│ ├── graph/ |
| 96 | +│ │ ├── node.py # Graph node representation |
| 97 | +│ │ ├── relationship.py # Graph relationship representation |
| 98 | +│ │ ├── pattern.py # Pattern matching |
| 99 | +│ │ └── database.py # In-memory graph database |
| 100 | +│ ├── io/ |
| 101 | +│ │ └── command_line.py # Interactive REPL |
| 102 | +│ ├── parsing/ |
| 103 | +│ │ ├── parser.py # Main parser |
| 104 | +│ │ ├── ast_node.py # AST node base class |
| 105 | +│ │ ├── expressions/ # Expression types (numbers, strings, operators) |
| 106 | +│ │ ├── functions/ # Built-in and custom functions |
| 107 | +│ │ ├── operations/ # Query operations (WITH, RETURN, UNWIND, etc.) |
| 108 | +│ │ ├── components/ # LOAD clause components |
| 109 | +│ │ ├── data_structures/ # Arrays, objects, lookups |
| 110 | +│ │ └── logic/ # CASE/WHEN/THEN/ELSE |
| 111 | +│ ├── tokenization/ |
| 112 | +│ │ ├── tokenizer.py # Lexer |
| 113 | +│ │ ├── token.py # Token class |
| 114 | +│ │ └── ... # Token types and mappers |
| 115 | +│ └── utils/ |
| 116 | +│ ├── string_utils.py # String manipulation utilities |
| 117 | +│ └── object_utils.py # Object utilities |
| 118 | +└── tests/ |
| 119 | + ├── test_extensibility.py |
| 120 | + ├── compute/ |
| 121 | + │ └── test_runner.py |
| 122 | + ├── graph/ |
| 123 | + │ ├── test_create.py |
| 124 | + │ ├── test_data.py |
| 125 | + │ └── test_match.py |
| 126 | + ├── parsing/ |
| 127 | + │ ├── test_parser.py |
| 128 | + │ ├── test_context.py |
| 129 | + │ └── test_expression.py |
| 130 | + └── tokenization/ |
| 131 | + ├── test_tokenizer.py |
| 132 | + ├── test_token_mapper.py |
| 133 | + └── test_trie.py |
| 134 | +``` |
| 135 | + |
| 136 | +## Creating Custom Functions |
| 137 | + |
| 138 | +```python |
| 139 | +from flowquery.extensibility import Function, FunctionDef |
| 140 | + |
| 141 | +@FunctionDef({ |
| 142 | + "description": "Converts a string to uppercase", |
| 143 | + "category": "string", |
| 144 | + "parameters": [ |
| 145 | + {"name": "text", "description": "String to convert", "type": "string"} |
| 146 | + ], |
| 147 | + "output": {"description": "Uppercase string", "type": "string"} |
| 148 | +}) |
| 149 | +class UpperCase(Function): |
| 150 | + def __init__(self): |
| 151 | + super().__init__("uppercase") |
| 152 | + self._expected_parameter_count = 1 |
| 153 | + |
| 154 | + def value(self) -> str: |
| 155 | + return str(self.get_children()[0].value()).upper() |
| 156 | +``` |
| 157 | + |
| 158 | +## License |
| 159 | + |
| 160 | +MIT License - see [LICENSE](LICENSE) for details. |
| 161 | + |
| 162 | +## Links |
| 163 | + |
| 164 | +- [Homepage](https://github.com/microsoft/FlowQuery/flowquery-py) |
| 165 | +- [Repository](https://github.com/microsoft/FlowQuery/flowquery-py) |
| 166 | +- [Issues](https://github.com/microsoft/FlowQuery/issues) |
0 commit comments