feat: support Python 3.10-3.14, drop 3.8/3.9

.claude/skills/plugin-test/SKILL.md

@@ -0,0 +1,222 @@
+---
+name: plugin-test
+description: Build Docker test images and run SkyWalking Python plugin/unit/e2e tests locally, mirroring the CI pipeline
+user-invocable: true
+---
+
+# Run Plugin Tests Locally
+
+This skill runs the same tests that CI runs, but locally. Ask the user what they want to test, or infer from recent changes.
+
+## Options
+
+The user can specify:
+- **Scope**: `all`, `unit`, a plugin name (e.g., `flask`, `redis`), a category (`data`, `http`, `web`), or `e2e`
+- **Python version**: e.g., `3.11` (default: `3.11-slim` for Docker image)
+- **Rebuild image**: whether to rebuild the Docker image (default: only if not already built)
+
+If the user just says `/plugin-test` with no args, ask what to test.
+
+## Prerequisites Check
+
+Before running tests, verify:
+```bash
+# Check Docker is running
+docker info > /dev/null 2>&1 || echo "ERROR: Docker is not running"
+
+# Check Poetry is available
+poetry --version || echo "ERROR: Poetry not found, run 'make poetry'"
+
+# Check if environment is set up
+poetry env info > /dev/null 2>&1 || echo "WARNING: Poetry env not set up, will run 'make env'"
+```
+
+If prerequisites fail, help the user fix them before proceeding.
+
+## Step 1: Setup Environment (if needed)
+
+```bash
+make env
+```
+
+This installs Poetry and all dependencies including dev and plugin groups.
+
+## Step 2: Build Plugin Test Docker Image
+
+The plugin test image must be built before running any plugin integration tests (not needed for unit tests).
+
+```bash
+# Default Python version
+docker build --build-arg BASE_PYTHON_IMAGE=3.11-slim \
+  -t apache/skywalking-python-agent:latest-plugin --no-cache \
+  . -f tests/plugin/Dockerfile.plugin
+```
+
+To test with a specific Python version:
+```bash
+docker build --build-arg BASE_PYTHON_IMAGE=3.12-slim \
+  -t apache/skywalking-python-agent:latest-plugin --no-cache \
+  . -f tests/plugin/Dockerfile.plugin
+```
+
+**Important**: The image tag is always `apache/skywalking-python-agent:latest-plugin` — this is what docker-compose files reference. Rebuilding with a different Python version replaces it.
+
+### Testing Multiple Python Versions
+
+The Python version **inside the Docker container** (not the host) determines what Python the plugin runs under. To test multiple versions, rebuild the image for each:
+
+```bash
+# Test with Python 3.11
+docker build --build-arg BASE_PYTHON_IMAGE=3.11-slim \
+  -t apache/skywalking-python-agent:latest-plugin --no-cache . \
+  -f tests/plugin/Dockerfile.plugin
+poetry run pytest -v tests/plugin/web/sw_flask/
+
+# Then test with Python 3.13
+docker build --build-arg BASE_PYTHON_IMAGE=3.13-slim \
+  -t apache/skywalking-python-agent:latest-plugin --no-cache . \
+  -f tests/plugin/Dockerfile.plugin
+poetry run pytest -v tests/plugin/web/sw_flask/
+```
+
+CI does this as a matrix (builds images for each Python version in parallel, then runs all tests against each). Locally, you rebuild and re-run sequentially.
+
+**Available Python base images**: `3.9-slim`, `3.10-slim`, `3.11-slim`, `3.12-slim`, `3.13-slim`, `3.14-slim`
+
+Check if image already exists:
+```bash
+docker images apache/skywalking-python-agent:latest-plugin --format "{{.ID}} {{.CreatedAt}}"
+```
+
+If it exists and is recent, ask the user if they want to rebuild or which Python version to use (rebuilding takes ~1-2 minutes).
+
+## Step 3: Run Tests
+
+### Unit Tests (no Docker needed)
+```bash
+poetry run pytest -v tests/unit/
+```
+
+### Single Plugin Test
+Map the plugin name to its test path:
+- Web frameworks: `tests/plugin/web/sw_<name>/`
+- HTTP clients: `tests/plugin/http/sw_<name>/`
+- Data stores: `tests/plugin/data/sw_<name>/`
+
+```bash
+poetry run pytest -v tests/plugin/web/sw_flask/
+poetry run pytest -v tests/plugin/data/sw_redis/
+poetry run pytest -v tests/plugin/http/sw_requests/
+```
+
+### Plugin Category
+```bash
+poetry run pytest -v tests/plugin/data/    # All data plugins
+poetry run pytest -v tests/plugin/http/    # All HTTP plugins
+poetry run pytest -v tests/plugin/web/     # All web plugins
+```
+
+### All Tests
+```bash
+poetry run pytest -v tests/unit/ tests/plugin/data/ tests/plugin/http/ tests/plugin/web/
+```
+
+### E2E Tests
+E2E tests need a separate Docker image and use SkyWalking infra-e2e:
+```bash
+# Build E2E image
+docker build --build-arg BASE_PYTHON_IMAGE=3.11-slim \
+  -t apache/skywalking-python-agent:latest-e2e --no-cache \
+  . -f tests/e2e/base/Dockerfile.e2e
+
+# E2E tests use infra-e2e framework, not pytest directly
+# They are defined in tests/e2e/case/*/e2e.yaml
+```
+
+Note: E2E tests require the `e2e` CLI tool from SkyWalking infra-e2e. They are typically only run in CI. Inform the user if they ask for E2E.
+
+## Step 4: Interpret Results
+
+### Success
+All tests pass with green output. Report the summary.
+
+### Failure - Docker Compose Timeout
+```
+Wait time exceeded 150 secs
+```
+This means services didn't start. Check:
+1. Is Docker running with enough resources?
+2. Is port 9090/9091/12800/19876 already in use? (`lsof -i :9090`)
+3. Check Docker logs: `docker compose -f tests/plugin/<category>/sw_<name>/docker-compose.yml logs`
+
+### Failure - Validation Mismatch
+The test prints a diff between expected and actual span data. Common causes:
+- Wrong `componentId` in expected.data.yml
+- Missing or extra spans
+- Tag values don't match
+- Operation name mismatch
+
+### Failure - Import/Install Error
+Plugin library failed to install in Docker. Check:
+- Library version compatibility with Python version
+- requirements.txt content in the test directory
+- Docker compose command for the service
+
+## Plugin Name to Test Path Mapping
+
+| Plugin | Test Path |
+|--------|-----------|
+| flask | tests/plugin/web/sw_flask/ |
+| django | tests/plugin/web/sw_django/ |
+| fastapi | tests/plugin/web/sw_fastapi/ |
+| sanic | tests/plugin/web/sw_sanic/ |
+| tornado | tests/plugin/web/sw_tornado/ |
+| bottle | tests/plugin/web/sw_bottle/ |
+| pyramid | tests/plugin/web/sw_pyramid/ |
+| falcon | tests/plugin/web/sw_falcon/ |
+| grpc | tests/plugin/web/sw_grpc/ |
+| requests | tests/plugin/http/sw_requests/ |
+| urllib3 | tests/plugin/http/sw_urllib3/ |
+| aiohttp | tests/plugin/http/sw_aiohttp/ |
+| httpx | tests/plugin/http/sw_httpx/ |
+| http | tests/plugin/http/sw_http/ |
+| http_wsgi | tests/plugin/http/sw_http_wsgi/ |
+| websockets | tests/plugin/http/sw_websockets/ |
+| redis | tests/plugin/data/sw_redis/ |
+| pymongo | tests/plugin/data/sw_pymongo/ |
+| pymysql | tests/plugin/data/sw_pymysql/ |
+| mysqlclient | tests/plugin/data/sw_mysqlclient/ |
+| psycopg | tests/plugin/data/sw_psycopg/ |
+| psycopg2 | tests/plugin/data/sw_psycopg2/ |
+| elasticsearch | tests/plugin/data/sw_elasticsearch/ |
+| kafka | tests/plugin/data/sw_kafka/ |
+| rabbitmq | tests/plugin/data/sw_rabbitmq/ |
+| pulsar | tests/plugin/data/sw_pulsar/ |
+| neo4j | tests/plugin/data/sw_neo4j/ |
+| happybase | tests/plugin/data/sw_happybase/ |
+| loguru | tests/plugin/data/sw_loguru/ |
+
+## Useful Debug Commands
+
+```bash
+# Check what containers are running during a test
+docker ps
+
+# Check container logs for a stuck test
+docker compose -f tests/plugin/<category>/sw_<name>/docker-compose.yml logs
+
+# Clean up orphaned containers from failed tests
+docker compose -f tests/plugin/<category>/sw_<name>/docker-compose.yml down --remove-orphans
+
+# Check what the mock collector received (while containers are running)
+curl http://localhost:12800/receiveData
+
+# Validate expected data manually
+curl -X POST http://localhost:12800/dataValidate -d @tests/plugin/<category>/sw_<name>/expected.data.yml
+
+# Check if ports are in use
+lsof -i :9090 -i :9091 -i :12800 -i :19876
+
+# Run with verbose output
+poetry run pytest -v -s tests/plugin/web/sw_flask/
+```

.github/workflows/CI.yaml

@@ -45,10 +45,12 @@ jobs:
       - name: Set up Python
         uses: actions/setup-python@v5
         with:
-          python-version: 3.8
+          python-version: "3.11"
       - name: Lint codes
         run: |
-          make env
+          make poetry
+          poetry install --all-extras --with lint
+          make gen
           make lint
 
   # Check if the plugin doc is generated correctly
@@ -65,7 +67,7 @@ jobs:
       - name: Set up Python
         uses: actions/setup-python@v5
         with:
-          python-version: 3.8
+          python-version: "3.14"
       - name: Check plugin doc
         run: |
           make env
@@ -86,10 +88,9 @@ jobs:
         with:
           persist-credentials: false
       - name: Check for file changes
-        uses: getsentry/paths-filter@v2.11.1
+        uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
         id: filter
         with:
-          token: ${{ github.token }}
           # The following filters indicate a category along with
           # the files that should not be ignored by CI when modified.
           filters: |
@@ -138,7 +139,7 @@ jobs:
     runs-on: ubuntu-latest
     strategy:
       matrix: # may support pypy in the future
-        python-version: [ "3.8-slim", "3.9-slim", "3.10-slim", "3.11-slim" ]
+        python-version: [ "3.10-slim", "3.11-slim", "3.12-slim", "3.13-slim", "3.14-slim" ]
       fail-fast: false
     env:
       BASE_PYTHON_IMAGE: ${{ matrix.python-version }}
@@ -171,7 +172,7 @@ jobs:
     timeout-minutes: 20
     strategy:
       matrix:
-        python-version: [ "3.8", "3.9", "3.10", "3.11" ]
+        python-version: [ "3.10", "3.11", "3.12", "3.13", "3.14" ]
         test-path: ${{fromJson(needs.prep-plugin-and-unit-tests.outputs.matrix)}}
       fail-fast: false
     env:
@@ -205,7 +206,9 @@ jobs:
           python-version: ${{ matrix.python-version }}
       - name: Run unit tests
         run: |
-          make env
+          make poetry
+          poetry install --all-extras --without plugins,lint
+          make gen
           poetry run pytest -v ${{ matrix.test-path }}
 
   docker-e2e:
@@ -219,7 +222,7 @@ jobs:
     timeout-minutes: 10
     strategy:
       matrix:
-        python-image-variant: [ "3.8-slim", "3.9-slim", "3.10-slim", "3.11-slim" ]
+        python-image-variant: [ "3.10-slim", "3.11-slim", "3.12-slim", "3.13-slim", "3.14-slim" ]
       fail-fast: false
     env:
       BASE_PYTHON_IMAGE: ${{ matrix.python-image-variant }}
@@ -251,7 +254,7 @@ jobs:
     timeout-minutes: 20
     strategy:
       matrix:
-        python-image-variant: [ "3.8-slim", "3.9-slim", "3.10-slim", "3.11-slim" ]
+        python-image-variant: [ "3.10-slim", "3.11-slim", "3.12-slim", "3.13-slim", "3.14-slim" ]
         case:
           - name: gRPC-single-process
             path: tests/e2e/case/grpc/single/e2e.yaml
@@ -275,6 +278,11 @@ jobs:
             path: tests/e2e/case/profiling/threading/e2e.yaml
           - name: profiling_greenlet
             path: tests/e2e/case/profiling/greenlet/e2e.yaml
+        exclude:
+          # gevent/greenlet does not support Python 3.14 yet
+          - python-image-variant: "3.14-slim"
+            case:
+              name: profiling_greenlet
       fail-fast: false
     steps:
       - name: Checkout source codes