Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -186,7 +186,7 @@
same "printed page" as the copyright notice for easier
identification within third-party archives.

Copyright 2025 Brian M. Dennis
Copyright 2025 - 2026 Brian M. Dennis

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
Expand Down
290 changes: 281 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,288 @@

[![QA](https://github.com/crossjam/scrobbledb/actions/workflows/qa.yml/badge.svg)](https://github.com/crossjam/scrobbledb/actions/workflows/qa.yml)

A modernization of [Jacob Kaplan-Moss’s](https://github.com/jacobian/)
project
[lastfm-to-sqlite](https://github.com/jacobian/lastfm-to-sqlite).
**Save your listening history from Last.fm or Libre.fm to a local SQLite database.**

Uses the [pylast](https://github.com/pylast/pylast) module underneath
the covers.
scrobbledb is a Python command-line tool that downloads your scrobble
data (listening history) from Last.fm or Libre.fm and stores it in a
SQLite database for local analysis, backup, and exploration. Built
with modern Python tooling, it offers rich terminal output, full-text
search, interactive browsing, and comprehensive data export
capabilities.

## Documentation
**Full disclosure, this project is primarily "auditionware".** The
main goal is to provide something for potential external collaborators
or employers to view and review. Yup, it’s a bit about me showing
off. If you have strong opinions feel free to fork this sucker and
take it where your heart desires.

See [docs/cli.md](docs/cli.md) for an overview of the
`scrobbledb` command line interface and links to per-command reference
pages in `docs/commands/`.
## About Last.fm and Scrobbling

[Last.fm](https://www.last.fm/) is a music discovery and tracking
service that records what you listen to across different platforms and
devices. This process of recording your listening history is called
"scrobbling."

**Scrobbling** automatically logs each track you play—including the
artist, album, track name, and timestamp—creating a detailed history
of your music consumption over time. Last.fm aggregates this data to
generate statistics, recommendations, and insights about your
listening habits.

The **Last.fm API** provides programmatic access to this scrobble
data, allowing applications like scrobbledb to retrieve and analyze
your complete listening history. [Libre.fm](https://libre.fm/) is an
open-source alternative that offers compatible scrobbling services.

## Why scrobbledb?

- **Local backup**: Keep your listening history in a local database you control
- **Advanced queries**: Use SQL to analyze your music habits in ways the web interface doesn't support
- **Data portability**: Export your data in multiple formats (JSON, CSV, TSV)
- **Full-text search**: Find tracks, albums, and artists instantly with SQLite FTS5
- **Interactive browsing**: Explore your library with a terminal UI
- **Privacy**: Your data stays on your machine

## Origin

scrobbledb is a modernization of [Jacob
Kaplan-Moss's](https://github.com/jacobian/)
[lastfm-to-sqlite](https://github.com/jacobian/lastfm-to-sqlite)
project. This version has been significantly expanded with:

- Modern Python tooling (uv, ruff, type hints)
- Domain-specific commands for exploring artists, albums, tracks, and plays
- Interactive terminal UI for browsing
- Full-text search capabilities
- Comprehensive data export options
- Enhanced statistics and filtering
- Rich terminal output with tables and progress bars

Original concept and implementation by Jacob Kaplan-Moss. Current development by Brian M. Dennis.

## Installation

scrobbledb requires Python 3.11 or later and uses [uv](https://github.com/astral-sh/uv) for dependency management.

```bash
# Clone the repository
git clone https://github.com/crossjam/scrobbledb.git
cd scrobbledb

# Install dependencies
uv sync

# Run scrobbledb
uv run scrobbledb --help
```

## Quick Start

### 1. Save your Last.fm credentials

```bash
uv run scrobbledb auth
```

This prompts for your Last.fm username, API key, shared secret, and
password, then saves them in `${APP_DIR}/auth.json`.

**Getting API credentials**: Visit [Last.fm
API](https://www.last.fm/api/account/create) to create an API account
and obtain your API key and shared secret.

### 2. Initialize the database

```bash
uv run scrobbledb config init
```

This creates the SQLite database and sets up the full-text search index.

### 3. Import your listening history

```bash
uv run scrobbledb ingest
```

This fetches your complete scrobble history from Last.fm and stores it
locally. Depending on how many scrobbles you have, this may take
several minutes.

### 4. Explore your data

```bash
# Search for tracks
uv run scrobbledb search "pink floyd"

# View recent plays
uv run scrobbledb plays list --limit 50

# Browse interactively
uv run scrobbledb browse

# See your top artists
uv run scrobbledb artists top --limit 20

# View statistics
uv run scrobbledb stats overview
```

## Command Overview

scrobbledb provides a comprehensive set of commands for managing and exploring your music data:

### Data Management

- **`auth`** - Configure Last.fm/Libre.fm API credentials ([docs](docs/commands/auth.md))
- **`config`** - Initialize database, reset data, or show configuration paths ([docs](docs/commands/config.md))
- **`ingest`** - Fetch listening history from Last.fm/Libre.fm ([docs](docs/commands/ingest.md))
- **`import`** - Import scrobbles from JSONL, CSV, or TSV files ([docs](docs/commands/import.md))
- **`index`** - Create or rebuild the full-text search index ([docs](docs/commands/index.md))
- **`export`** - Export data in various formats with presets or custom SQL ([docs](docs/commands/export.md))

### Data Exploration

- **`search`** - Full-text search across artists, albums, and tracks ([docs](docs/commands/search.md))
- **`browse`** - Interactive terminal UI for browsing tracks ([docs](docs/commands/browse.md))
- **`plays`** - View and filter listening history chronologically ([docs](docs/commands/plays.md))
- **`artists`** - Browse artists, view top artists, see detailed statistics ([docs](docs/commands/artists.md))
- **`albums`** - Search albums and view album details with tracks ([docs](docs/commands/albums.md))
- **`tracks`** - Search tracks, view top tracks, see play history ([docs](docs/commands/tracks.md))
- **`stats`** - Generate listening statistics (overview, monthly, yearly) ([docs](docs/commands/stats.md))

### Advanced

- **`sql`** - Direct access to sqlite-utils commands for power users ([docs](docs/commands/sql.md))
- **`version`** - Display the installed version ([docs](docs/commands/version.md))

See the [CLI overview](docs/cli.md) for a complete command reference and detailed documentation for each command.

## Database Schema

scrobbledb stores your data in a normalized SQLite database:

- **`artists`** - Artist information (id, name)
- **`albums`** - Album information (id, title, artist_id)
- **`tracks`** - Track information (id, title, album_id)
- **`plays`** - Play events (track_id, timestamp)
- **`tracks_fts`** - FTS5 full-text search index

This schema enables efficient queries, comprehensive searches, and detailed analysis of your listening history.

## Example Workflows

### Backup your data weekly

```bash
# Update with new scrobbles
uv run scrobbledb ingest --since-date "7 days ago"

# Export to JSON backup
uv run scrobbledb export plays --format json --output backup-$(date +%Y%m%d).json
```

### Find your most-played tracks from a specific year

```bash
uv run scrobbledb tracks top --since 2023-01-01 --until 2023-12-31 --limit 50
```

### Analyze your listening patterns

```bash
# Monthly breakdown
uv run scrobbledb stats monthly --year 2024

# Top artists in the last 30 days
uv run scrobbledb artists top --since "30 days ago"

# All plays for a specific artist
uv run scrobbledb plays list --artist "Radiohead" --limit 1000
```

### Export data for external analysis

```bash
# Export to CSV for Excel/pandas
uv run scrobbledb export plays --format csv --output plays.csv

# Custom SQL query
uv run scrobbledb export --sql "SELECT artist_name, COUNT(*) as plays FROM plays GROUP BY artist_name" --format csv
```

## Configuration

scrobbledb follows the XDG Base Directory specification. By default, data is stored in:

- **Linux/Unix**: `~/.local/share/dev.pirateninja.scrobbledb/`
- **macOS**: `~/Library/Application Support/dev.pirateninja.scrobbledb/`
- **Windows**: `%LOCALAPPDATA%\dev.pirateninja.scrobbledb\`

You can override the database and auth file locations using command-line options:

```bash
uv run scrobbledb --database /path/to/custom.db ingest --auth /path/to/auth.json
```

## Development

scrobbledb uses modern Python development tools:

- **uv** - Fast Python package manager
- **ruff** - Fast Python linter and formatter
- **pytest** - Testing framework
- **poe** - Task runner for common development tasks

```bash
# Run tests
poe test

# Lint code
poe lint

# Type check
poe type

# Run all quality checks
poe qa
```

## Agentic Coding Disclosure

Significant portions of this project were implemented through the use
of agentic coding tools such as Claude Code, GitHub Copilot Agent,
OpenAI Codex, and Gemini CLI. This was a specific goal intended to
explore and increase my proficiency with AI accelerated coding
practices.

See [AGENTS.md](AGENTS.md) for detailed development guidelines.

## Contributing

**As mentioned above, this project is primarily "auditionware".**

However, pull requests and issues are welcome, at least as criticism,
feedback, and inspiration! There might be a lag on responding or
acceptance though. You’re likely best off assuming that a PR will take
forever to be accepted if at all. Similarly for addressing issues. For
major changes, please open an issue first to discuss what you would
like to change.

## License

scrobbledb is licensed under the Apache License 2.0. See [LICENSE](LICENSE) for the full license text.

Original lastfm-to-sqlite project by Jacob Kaplan-Moss, originally
under the [WTFPL][wtfpl] based upon his repository’s [`pyproject.toml`][jacobian-pyproject-toml]
file.

## Links

- **Repository**: https://github.com/crossjam/scrobbledb
- **Original Project**: https://github.com/jacobian/lastfm-to-sqlite
- **Last.fm API**: https://www.last.fm/api
- **Libre.fm**: https://libre.fm/

[wtfpl]: https://www.wtfpl.net
[jacobian-pyproject-toml]: https://github.com/jacobian/lastfm-to-sqlite/blob/8118b453b36142241618c484cf74c7916423f649/pyproject.toml
12 changes: 6 additions & 6 deletions docs/commands/albums.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,11 @@ Album investigation commands. Search for albums and view detailed information ab
from click.testing import CliRunner
from scrobbledb.cli import cli
runner = CliRunner()
result = runner.invoke(cli, ["albums", "--help"])
result = runner.invoke(cli, ["albums", "--help"], prog_name='scrobbledb')
cog.out("```\n" + result.output + "```")
]]] -->
```
Usage: cli albums [OPTIONS] COMMAND [ARGS]...
Usage: scrobbledb albums [OPTIONS] COMMAND [ARGS]...

Album investigation commands.

Expand All @@ -34,11 +34,11 @@ Commands:
Search for albums using fuzzy matching. Find albums by partial name when you don't remember exact titles.

<!-- [[[cog
result = runner.invoke(cli, ["albums", "search", "--help"])
result = runner.invoke(cli, ["albums", "search", "--help"], prog_name='scrobbledb')
cog.out("```\n" + result.output + "```")
]]] -->
```
Usage: cli albums search [OPTIONS] QUERY
Usage: scrobbledb albums search [OPTIONS] QUERY

Search for albums using fuzzy matching.

Expand Down Expand Up @@ -72,11 +72,11 @@ Options:
Display detailed information about a specific album and list its tracks with play statistics.

<!-- [[[cog
result = runner.invoke(cli, ["albums", "show", "--help"])
result = runner.invoke(cli, ["albums", "show", "--help"], prog_name='scrobbledb')
cog.out("```\n" + result.output + "```")
]]] -->
```
Usage: cli albums show [OPTIONS] [ALBUM_TITLE]
Usage: scrobbledb albums show [OPTIONS] [ALBUM_TITLE]

Display detailed information about a specific album and list its tracks.

Expand Down
Loading
Loading