diff --git a/README.md b/README.md index 97556117..4cc828d5 100644 --- a/README.md +++ b/README.md @@ -39,7 +39,7 @@ AI-Driven Development (vibe coding) on Databricks just got a whole lot better. T | [**Visual Builder App**](#visual-builder-app) | Web-based UI for Databricks development | `databricks-builder-app/` | | [**Core Library**](#core-library) | Building custom integrations (LangChain, OpenAI, etc.) | `pip install` | | [**Skills Only**](databricks-skills/) | Provide Databricks patterns and best practices (without MCP functions) | Install skills | -| [**Genie Code Skills**](databricks-skills/install_skills_to_genie_code.sh) | Install Databricks skills for Genie Code to reference | [Genie Code skills (install)](#genie-code-skills) | +| [**Genie Code Skills**](databricks-skills/install_skills.sh) | Install skills into your workspace for Genie Code (`--install-to-genie`) | [Genie Code skills (install)](#genie-code-skills) | | [**MCP Tools Only**](databricks-mcp-server/) | Just executable actions (no guidance) | Register MCP server | --- @@ -160,23 +160,32 @@ Works with LangChain, OpenAI Agents SDK, or any Python framework. See [databrick --- ## Genie Code Skills - - Will install and deploy all available skills to your personal skills directory for all Genie Code sessions to reference while planning/building anything directly in the UI. No post-install steps as workspace is automatically configured during install process for Genie Code to use the skills. - **Basic installation** (uses DEFAULT profile) +Install skills into `./.claude/skills` (relative to the directory where you run the script), then upload them to your workspace at `/Workspace/Users//.assistant/skills` so Genie Code can use them in the UI. Requires the [Databricks CLI](https://docs.databricks.com/aws/en/dev-tools/cli/) authenticated for your workspace. + +**Always run from the project directory** where you want `.claude/skills` created (for example your repo root or `ai-dev-kit`). + +**From this repo (recommended if you have a clone):** ```bash -#Execute from root folder (/ai-dev-kit) -./databricks-skills/install_skills_to_genie_code.sh +# Databricks skills from this checkout + upload (DEFAULT CLI profile) +./databricks-skills/install_skills.sh --local --install-to-genie + +# Download all skills from GitHub, then upload +./databricks-skills/install_skills.sh --install-to-genie + +# Explicit Databricks CLI profile +./databricks-skills/install_skills.sh --install-to-genie --profile YOUR_PROFILE ``` -**Advance installation** (uses provided profile) +**Without cloning** (run from the directory that should contain `.claude/skills`): ```bash -#Execute from root folder (/ai-dev-kit) -./databricks-skills/install_skills_to_genie_code +curl -sSL https://raw.githubusercontent.com/databricks-solutions/ai-dev-kit/main/databricks-skills/install_skills.sh | bash -s -- --install-to-genie ``` +Combine `--profile`, `--local`, specific skill names, `--mlflow-version`, etc. as needed; see `./databricks-skills/install_skills.sh --help` or [databricks-skills/README.md](databricks-skills/README.md). + **Skill modification or Custom Skill** After the script successfully installs the skills to your workspace, you may find the skills under `/Workspace/Users//.assistant/skills`. diff --git a/databricks-skills/README.md b/databricks-skills/README.md index 1d91ae49..46d52dc2 100644 --- a/databricks-skills/README.md +++ b/databricks-skills/README.md @@ -4,22 +4,61 @@ Skills that teach Claude Code how to work effectively with Databricks - providin ## Installation -Run in your project root: +Run from your **project root** (the directory where you want `.claude/skills` created). + +### From this repository (local script) + +If you already have the repo (fork or clone), use the script on disk: ```bash -# Install all skills (Databricks + MLflow) -curl -sSL https://raw.githubusercontent.com/databricks-solutions/ai-dev-kit/main/databricks-skills/install_skills.sh | bash +# Install all skills (Databricks + MLflow + APX) — downloads from GitHub by default +./databricks-skills/install_skills.sh + +# Install Databricks skills only from this checkout (no network for those skills) +./databricks-skills/install_skills.sh --local # Install specific skills +./databricks-skills/install_skills.sh databricks-bundles agent-evaluation + +# Pin MLflow / APX versions +./databricks-skills/install_skills.sh --mlflow-version v1.0.0 + +# List available skills +./databricks-skills/install_skills.sh --list + +# Install + upload to workspace for Genie Code (/Workspace/Users//.assistant/skills) +./databricks-skills/install_skills.sh --install-to-genie + +./databricks-skills/install_skills.sh --install-to-genie --profile prod + +# Local Databricks skills + Genie upload +./databricks-skills/install_skills.sh --local --install-to-genie +``` + +Paths assume you are at the **ai-dev-kit** repo root. From another project, copy or symlink the script, or use the `curl` flow below. + +### Without cloning (curl) + +Use this when you only want the installer and not the full repo: + +```bash +# Install all skills +curl -sSL https://raw.githubusercontent.com/databricks-solutions/ai-dev-kit/main/databricks-skills/install_skills.sh | bash + +# Install specific skills (pass args after bash -s --) curl -sSL https://raw.githubusercontent.com/databricks-solutions/ai-dev-kit/main/databricks-skills/install_skills.sh | bash -s -- databricks-bundles agent-evaluation -# Pin MLflow skills to a specific version curl -sSL https://raw.githubusercontent.com/databricks-solutions/ai-dev-kit/main/databricks-skills/install_skills.sh | bash -s -- --mlflow-version v1.0.0 -# List available skills curl -sSL https://raw.githubusercontent.com/databricks-solutions/ai-dev-kit/main/databricks-skills/install_skills.sh | bash -s -- --list + +curl -sSL https://raw.githubusercontent.com/databricks-solutions/ai-dev-kit/main/databricks-skills/install_skills.sh | bash -s -- --install-to-genie + +curl -sSL https://raw.githubusercontent.com/databricks-solutions/ai-dev-kit/main/databricks-skills/install_skills.sh | bash -s -- --install-to-genie --profile prod ``` +`--install-to-genie` uploads the tree under `./.claude/skills` to the workspace (requires the `databricks` CLI). + This creates `.claude/skills/` and downloads all skills. Claude Code loads them automatically. - **Databricks skills** are downloaded from this repository - **MLflow skills** are fetched dynamically from [github.com/mlflow/skills](https://github.com/mlflow/skills) diff --git a/databricks-skills/install_skills.sh b/databricks-skills/install_skills.sh index 7d842b6b..7630615c 100755 --- a/databricks-skills/install_skills.sh +++ b/databricks-skills/install_skills.sh @@ -17,6 +17,8 @@ # ./install_skills.sh databricks-bundles agent-evaluation # Install specific skills # ./install_skills.sh --mlflow-version v1.0.0 # Pin MLflow skills version # ./install_skills.sh --local # Install Databricks skills from local directory +# ./install_skills.sh --install-to-genie # Install then upload ./.claude/skills to workspace (Genie Code / Assistant) +# ./install_skills.sh --install-to-genie --profile prod --local # ./install_skills.sh --list # List available skills # ./install_skills.sh --help # Show help # @@ -35,6 +37,9 @@ REPO_URL="https://github.com/databricks-solutions/ai-dev-kit" REPO_RAW_URL="https://raw.githubusercontent.com/databricks-solutions/ai-dev-kit/main" SKILLS_DIR=".claude/skills" INSTALL_FROM_LOCAL=false +INSTALL_TO_GENIE=false +# Databricks CLI profile for workspace upload (only used with --install-to-genie) +DB_PROFILE="${DATABRICKS_CONFIG_PROFILE:-DEFAULT}" SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" # MLflow skills configuration @@ -180,6 +185,9 @@ show_help() { echo " --list, -l List all available skills" echo " --all, -a Install all skills (default if no skills specified)" echo " --local Install from local files instead of downloading" + echo " --install-to-genie After install, upload ./.claude/skills to workspace" + echo " /Users//.assistant/skills for Genie Code (uses cwd; requires databricks CLI)" + echo " --profile Databricks config profile for workspace upload (default: DEFAULT or \$DATABRICKS_CONFIG_PROFILE)" echo " --mlflow-version Pin MLflow skills to specific version/branch/tag (default: main)" echo " --apx-version Pin APX skills to specific version/branch/tag (default: main)" echo "" @@ -191,6 +199,8 @@ show_help() { echo " ./install_skills.sh --mlflow-version v1.0.0 # Pin MLflow skills version" echo " ./install_skills.sh --apx-version v1.0.0 # Pin APX skills version" echo " ./install_skills.sh --local # Install all from local directory" + echo " ./install_skills.sh --install-to-genie # Install all, then upload to workspace for Genie Code" + echo " ./install_skills.sh --install-to-genie --profile prod # Same with explicit Databricks CLI profile" echo " ./install_skills.sh --list # List available skills" echo "" echo -e "${GREEN}Databricks Skills:${NC}" @@ -245,6 +255,93 @@ is_valid_skill() { return 1 } +# Upload one skill folder to workspace /Users//.assistant/skills// +upload_skill_to_genie_workspace() { + local skill_dir="$1" + local skills_path="$2" + local db_profile="$3" + + skill_dir="${skill_dir%/}" + local skill_name + skill_name=$(basename "$skill_dir") + + if [[ "$skill_name" == "."* ]] || [[ "$skill_name" == "TEMPLATE" ]] || [[ ! -d "$skill_dir" ]]; then + return 0 + fi + if [[ ! -f "$skill_dir/SKILL.md" ]]; then + return 0 + fi + + echo -e " ${GREEN}Uploading${NC} $skill_name" + databricks workspace mkdirs "$skills_path/$skill_name" --profile "$db_profile" 2>/dev/null || true + + while IFS= read -r -d '' file; do + rel_path="${file#$skill_dir/}" + dest_path="$skills_path/$skill_name/$rel_path" + parent_dir=$(dirname "$dest_path") + if [[ "$parent_dir" != "$skills_path/$skill_name" ]]; then + databricks workspace mkdirs "$parent_dir" --profile "$db_profile" 2>/dev/null || true + fi + databricks workspace import "$dest_path" --file "$file" --profile "$db_profile" --format AUTO --overwrite 2>/dev/null || true + done < <(find "$skill_dir" -type f \( -name "*.md" -o -name "*.py" -o -name "*.yaml" -o -name "*.yml" -o -name "*.sh" \) -print0) +} + +# Upload all skills under SKILLS_DIR to workspace for Genie Code / Assistant agent mode +install_skills_to_genie_workspace() { + if ! command -v databricks >/dev/null 2>&1; then + echo -e "${RED}Error: databricks CLI not found. Install it to use --install-to-genie.${NC}" + return 1 + fi + + local abs_skills_dir + if [ ! -d "$SKILLS_DIR" ]; then + echo -e "${RED}Error: Skills directory not found: ${SKILLS_DIR} (run from the directory where skills were installed).${NC}" + return 1 + fi + abs_skills_dir="$(cd "$SKILLS_DIR" && pwd)" + + echo "" + echo -e "${BLUE}════════════════════════════════════════════════════════════${NC}" + echo -e "${BLUE} Uploading skills to workspace (Genie Code)${NC}" + echo -e "${BLUE}════════════════════════════════════════════════════════════${NC}" + echo -e "Databricks profile: ${DB_PROFILE}" + echo -e "Local skills (cwd-relative): ${SKILLS_DIR}/ → ${abs_skills_dir}" + echo "" + + local user_name + user_name=$(databricks current-user me --profile "$DB_PROFILE" --output json 2>/dev/null | python3 -c "import sys, json; d=json.load(sys.stdin); print(d.get('userName', ''))" 2>/dev/null || echo "") + if [ -z "$user_name" ]; then + echo -e "${RED}Error: Could not determine workspace user. Check authentication and --profile.${NC}" + return 1 + fi + + local skills_path="/Users/$user_name/.assistant/skills" + echo -e "Workspace user: ${user_name}" + echo -e "Workspace path: ${skills_path}" + echo "" + + echo -e "${GREEN}Creating workspace skills directory...${NC}" + databricks workspace mkdirs "$skills_path" --profile "$DB_PROFILE" 2>/dev/null || true + + echo -e "${GREEN}Uploading skills...${NC}" + local skill_dir + for skill_dir in "$abs_skills_dir"/*/; do + [ -d "$skill_dir" ] || continue + upload_skill_to_genie_workspace "$skill_dir" "$skills_path" "$DB_PROFILE" + done + + echo "" + echo -e "${GREEN}Workspace listing:${NC}" + databricks workspace list "$skills_path" --profile "$DB_PROFILE" 2>/dev/null || echo -e " ${YELLOW}(Could not list workspace path)${NC}" + + local skills_count + skills_count=$(find "$abs_skills_dir" -maxdepth 1 -type d -exec test -f {}/SKILL.md \; -print 2>/dev/null | wc -l | tr -d ' ') + echo "" + echo -e "${GREEN}Genie Code upload complete.${NC} ${skills_count} skills under ${skills_path}" + echo "" + return 0 +} + # Function to download a Databricks skill download_databricks_skill() { local skill_name=$1 @@ -455,6 +552,18 @@ while [ $# -gt 0 ]; do INSTALL_FROM_LOCAL=true shift ;; + --install-to-genie|--deploy-to-assistant) + INSTALL_TO_GENIE=true + shift + ;; + --profile) + if [ -z "$2" ] || [ "${2:0:1}" = "-" ]; then + echo -e "${RED}Error: --profile requires a profile name${NC}" + exit 1 + fi + DB_PROFILE="$2" + shift 2 + ;; --mlflow-version) if [ -z "$2" ] || [ "${2:0:1}" = "-" ]; then echo -e "${RED}Error: --mlflow-version requires a version/ref argument${NC}" @@ -567,3 +676,7 @@ for skill in $SKILLS_TO_INSTALL; do fi done +if [ "$INSTALL_TO_GENIE" = true ]; then + install_skills_to_genie_workspace || exit 1 +fi + diff --git a/databricks-skills/install_skills_to_genie_code.sh b/databricks-skills/install_skills_to_genie_code.sh deleted file mode 100755 index 8141044f..00000000 --- a/databricks-skills/install_skills_to_genie_code.sh +++ /dev/null @@ -1,119 +0,0 @@ -#!/bin/bash -# Deploy skills to Databricks Assistant skills folder -# This enables the built-in Assistant in Agent mode to use all skills -# -# PREREQUISITE: Run ./install_skills.sh first to install skills locally -# -# Usage: ./install_to_dbx_assistant.sh - -set -e - -PROFILE=${1:-"DEFAULT"} -SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" -PROJECT_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)" -LOCAL_SKILLS_DIR="$PROJECT_ROOT/.claude/skills" - -echo "================================================" -echo "Deploying Skills to Databricks Assistant" -echo "================================================" -echo "Profile: $PROFILE" -echo "" - -# Install all skills (databricks + MLflow + APX) via install_skills.sh -echo " Installing all skills via install_skills.sh..." -INSTALL_SKILLS_SCRIPT="$PROJECT_ROOT/databricks-skills/install_skills.sh" -if [ ! -f "$INSTALL_SKILLS_SCRIPT" ]; then - echo -e "${RED}Error: install_skills.sh not found at ${INSTALL_SKILLS_SCRIPT}${NC}" - exit 1 -fi - -# Run install_skills.sh to download all skills (databricks, MLflow, APX) -(bash "$INSTALL_SKILLS_SCRIPT") - -# Get current user email -USER_EMAIL=$(databricks current-user me --profile "$PROFILE" --output json 2>/dev/null | python3 -c "import sys, json; d=json.load(sys.stdin); print(d.get('userName', ''))" 2>/dev/null || echo "") -if [ -z "$USER_EMAIL" ]; then - echo "Error: Could not determine user email." - exit 1 -fi - -# Skills destination in workspace -SKILLS_PATH="/Users/$USER_EMAIL/.assistant/skills" - -echo "User: $USER_EMAIL" -echo "Skills Path: $SKILLS_PATH" -echo "" - -# Create the skills directory -echo "Step 1: Creating skills directory..." -databricks workspace mkdirs "$SKILLS_PATH" --profile "$PROFILE" 2>/dev/null || true -echo "" - -# Function to upload a skill directory -upload_skill() { - local skill_dir="$1" - # Remove trailing slash if present - skill_dir="${skill_dir%/}" - local skill_name=$(basename "$skill_dir") - - # Skip hidden dirs, TEMPLATE, and non-skill directories - if [[ "$skill_name" == "."* ]] || [[ "$skill_name" == "TEMPLATE" ]] || [[ ! -d "$skill_dir" ]]; then - return - fi - - # Check if it has a SKILL.md - if [[ ! -f "$skill_dir/SKILL.md" ]]; then - return - fi - - echo " Uploading skill: $skill_name" - - # Create skill directory in workspace - databricks workspace mkdirs "$SKILLS_PATH/$skill_name" --profile "$PROFILE" 2>/dev/null || true - - # Upload all markdown, python, and yaml files in the skill directory - find "$skill_dir" -type f \( -name "*.md" -o -name "*.py" -o -name "*.yaml" -o -name "*.yml" -o -name "*.sh" \) | while read -r file; do - # Get the relative path within the skill directory - rel_path="${file#$skill_dir/}" - dest_path="$SKILLS_PATH/$skill_name/$rel_path" - - # Create parent directories if needed (for subdirs like scripts/) - parent_dir=$(dirname "$dest_path") - if [[ "$parent_dir" != "$SKILLS_PATH/$skill_name" ]]; then - databricks workspace mkdirs "$parent_dir" --profile "$PROFILE" 2>/dev/null || true - fi - - # Upload file - databricks workspace import "$dest_path" --file "$file" --profile "$PROFILE" --format AUTO --overwrite 2>/dev/null || true - done -} - -# Upload all skills from local .claude/skills directory -echo "Step 2: Uploading skills from $LOCAL_SKILLS_DIR..." -for skill_dir in "$LOCAL_SKILLS_DIR"/*/; do - upload_skill "$skill_dir" -done -echo "" - -# List uploaded skills -echo "Step 3: Verifying deployment..." -echo "" -databricks workspace list "$SKILLS_PATH" --profile "$PROFILE" 2>/dev/null || echo " (Could not list skills)" -echo "" - -echo "================================================" -echo "Deployment complete!" -echo "" -echo "Your Databricks Assistant in Agent mode now has access to:" -echo "" -skills_count=$(find "$LOCAL_SKILLS_DIR" -maxdepth 1 -type d -exec test -f {}/SKILL.md \; -print 2>/dev/null | wc -l | tr -d ' ') -echo " $skills_count skills deployed to: $SKILLS_PATH" -echo "" -echo "To use:" -echo " 1. Open a Databricks notebook/sql query/anywhere in the workspace" -echo " 2. Open the Genie Code panel" -echo " 3. Switch to 'Agent' mode" -echo " 4. Ask: 'Create a dashboard for my sales data'" -echo "" -echo "The Assistant will automatically load relevant skills!" -echo "================================================"