jessikitty
89f65e2d74
Downloads official instruction PDFs from LEGO, located via the Brickset API, one set per interval (10 minutes by default) so neither API gets hammered. Official sets only; MOCs are excluded throughout. - New admin page at /admin/instruction-fetch: queue builder, pacing controls, live countdown and per-set job history - Database-backed queue (instruction_fetch_jobs) and last-run timestamp, so pacing survives restarts - Background worker thread, plus 'flask instructions queue/run/status' for Task Scheduler or cron - SHA-256 dedupe (instructions.sha256), PDF header check, size cap and streaming downloads via temp files - brickset_api: getInstructions2 by set number and getInstructions by setID (the old code sent a set number to the setID method); getSets params now proper JSON; getKeyUsageStats added - Schema top-ups applied automatically at boot: new table plus instructions.sha256/source - Docker: fetcher env vars passed through compose, no worker thread in the entrypoint bootstrap
298 lines
7.9 KiB
Markdown
298 lines
7.9 KiB
Markdown
# 🧱 LEGO Instructions Manager
|
|
|
|
A Python web application for organizing and managing your LEGO instruction manuals. Upload PDFs and images, search by theme, set number, or year, and integrate with Brickset API for automatic set details.
|
|
|
|
|
|
|
|
|
|
|
|
## ✨ Features
|
|
|
|
- 📤 **Upload & Organize**: Upload instruction PDFs and images for your LEGO sets
|
|
- 🔍 **Advanced Search**: Search and filter by theme, year, set number, or name
|
|
- 🎨 **Theme Organization**: Organize sets by LEGO themes automatically
|
|
- 🔗 **Brickset Integration**: Auto-populate set details using Brickset API v3
|
|
- 👤 **User Authentication**: Secure login and user management
|
|
- 📱 **Responsive Design**: Works on desktop, tablet, and mobile
|
|
- 📊 **Statistics Dashboard**: View collection statistics and insights
|
|
- 🖼️ **Image Gallery**: View instruction images in an organized gallery
|
|
- 📄 **PDF Support**: View and download PDF instructions directly
|
|
- 💾 **Backup & Restore**: One-click archive of the database plus every uploaded file, ready to restore anywhere
|
|
- ⬇️ **Automatic Instruction Fetcher**: Downloads official instruction PDFs from LEGO (found via Brickset) for official sets, one every 10 minutes so nobody's API gets hammered — see [docs/INSTRUCTION_FETCHER.md](docs/INSTRUCTION_FETCHER.md)
|
|
|
|
## 🚀 Quick Start
|
|
|
|
### Prerequisites
|
|
|
|
- Python 3.8 or higher
|
|
- pip (Python package manager)
|
|
- Git
|
|
|
|
### Installation
|
|
|
|
1. **Clone the repository**
|
|
```bash
|
|
git clone https://gitea.hideawaygaming.com.au/jessikitty/lego-instructions-manager.git
|
|
cd lego-instructions-manager
|
|
```
|
|
|
|
2. **Create a virtual environment**
|
|
```bash
|
|
python3 -m venv venv
|
|
source venv/bin/activate # On Windows: venv\Scripts\activate
|
|
```
|
|
|
|
3. **Install dependencies**
|
|
```bash
|
|
pip install -r requirements.txt
|
|
```
|
|
|
|
4. **Configure environment variables**
|
|
```bash
|
|
cp .env.example .env
|
|
# Edit .env with your settings
|
|
```
|
|
|
|
5. **Initialize the database**
|
|
```bash
|
|
flask --app run.py init-db
|
|
```
|
|
|
|
6. **Create an admin user**
|
|
```bash
|
|
flask --app run.py create-admin
|
|
```
|
|
|
|
7. **Run the application**
|
|
```bash
|
|
python run.py
|
|
```
|
|
|
|
The application will be available at `http://localhost:5000`
|
|
|
|
## ⚙️ Configuration
|
|
|
|
### Environment Variables
|
|
|
|
Edit the `.env` file with your configuration:
|
|
|
|
```env
|
|
# Flask Configuration
|
|
SECRET_KEY=your-secret-key-here
|
|
FLASK_ENV=development
|
|
|
|
# Database (SQLite by default)
|
|
DATABASE_URL=sqlite:///lego_instructions.db
|
|
|
|
# Brickset API (optional but recommended)
|
|
BRICKSET_API_KEY=your-api-key
|
|
BRICKSET_USERNAME=your-username
|
|
BRICKSET_PASSWORD=your-password
|
|
|
|
# Upload Settings
|
|
MAX_CONTENT_LENGTH=52428800 # 50MB
|
|
SETS_PER_PAGE=20
|
|
|
|
# Backups
|
|
BACKUP_FOLDER=backups
|
|
BACKUP_KEEP=5
|
|
```
|
|
|
|
### Brickset API Setup
|
|
|
|
1. Register for a free account at [Brickset.com](https://brickset.com)
|
|
2. Request an API key at [Brickset API Documentation](https://brickset.com/article/52664/api-version-3-documentation)
|
|
3. Add your credentials to the `.env` file
|
|
|
|
The application works without Brickset, but you'll miss out on automatic set data population.
|
|
|
|
## 📖 Usage
|
|
|
|
### Adding a Set
|
|
|
|
1. Click "Add Set" in the navigation
|
|
2. Enter set details manually OR search using Brickset
|
|
3. Save the set to your collection
|
|
|
|
### Uploading Instructions
|
|
|
|
1. Navigate to a set's detail page
|
|
2. Click "Upload Instructions"
|
|
3. Select PDF or image files
|
|
4. Files are automatically organized by set number
|
|
|
|
### Searching & Filtering
|
|
|
|
- Use the search bar to find sets by name or number
|
|
- Filter by theme or year using the dropdowns
|
|
- Sort by set number, name, theme, year, or newest first
|
|
|
|
## 🗂️ Project Structure
|
|
|
|
```
|
|
lego-instructions-manager/
|
|
├── app/
|
|
│ ├── models/ # Database models
|
|
│ ├── routes/ # Application routes/views
|
|
│ ├── services/ # Business logic (API, file handling, backups)
|
|
│ ├── templates/ # HTML templates
|
|
│ ├── static/ # CSS, JS, uploaded files
|
|
│ └── cli.py # flask backup commands
|
|
├── backups/ # Backup archives (git-ignored)
|
|
├── docs/ # Documentation
|
|
├── migrations/ # Database migrations
|
|
├── tests/ # Unit tests
|
|
├── requirements.txt # Python dependencies
|
|
├── run.py # Application entry point
|
|
└── README.md # This file
|
|
```
|
|
|
|
## 🛠️ Development
|
|
|
|
### Database Migrations
|
|
|
|
When you modify database models:
|
|
|
|
```bash
|
|
flask --app run.py db init # Initialize migrations (first time only)
|
|
flask --app run.py db migrate -m "Description" # Create migration
|
|
flask --app run.py db upgrade # Apply migration
|
|
```
|
|
|
|
### Running Tests
|
|
|
|
```bash
|
|
pytest tests/
|
|
```
|
|
|
|
### Flask Shell
|
|
|
|
Access the Flask shell for debugging:
|
|
|
|
```bash
|
|
flask --app run.py shell
|
|
```
|
|
|
|
## 📊 Database Schema
|
|
|
|
### User
|
|
- id, username, email, password_hash, is_admin, created_at
|
|
|
|
### Set
|
|
- id, set_number, set_name, theme, year_released, piece_count
|
|
- is_moc, moc_designer, moc_description
|
|
- brickset_id, image_url, cover_image, user_id, created_at, updated_at
|
|
|
|
### Instruction
|
|
- id, set_id, file_type, file_path, file_name, file_size, thumbnail_path
|
|
- page_number, user_id, uploaded_at
|
|
- sha256, source (set by the automatic fetcher; null for manual uploads)
|
|
|
|
### InstructionFetchJob
|
|
- id, set_id, status, attempts, files_added, source, message
|
|
- queued_at, not_before, started_at, finished_at, queued_by_id
|
|
|
|
### ExtraFile
|
|
- id, set_id, file_name, original_filename, file_path, file_type, file_size
|
|
- description, category, uploaded_at, uploaded_by
|
|
|
|
## 🔒 Security Notes
|
|
|
|
- Passwords are hashed using bcrypt
|
|
- File uploads are validated and sanitized
|
|
- User authentication required for all operations
|
|
- CSRF protection enabled on all forms
|
|
|
|
## 🚢 Deployment
|
|
|
|
### Production Considerations
|
|
|
|
1. **Use PostgreSQL** instead of SQLite:
|
|
```env
|
|
DATABASE_URL=postgresql://user:password@localhost/lego_db
|
|
```
|
|
|
|
2. **Set a strong SECRET_KEY**:
|
|
```python
|
|
import secrets
|
|
print(secrets.token_hex(32))
|
|
```
|
|
|
|
3. **Use gunicorn** for production:
|
|
```bash
|
|
gunicorn -w 4 -b 0.0.0.0:8000 "run:app"
|
|
```
|
|
|
|
4. **Set up nginx** as a reverse proxy
|
|
5. **Enable HTTPS** with Let's Encrypt
|
|
6. **Regular backups** — see below
|
|
|
|
## 💾 Backup & Restore
|
|
|
|
Full in-place backup of the database plus every uploaded PDF, image, cover,
|
|
thumbnail and extra file — into a single verifiable `.zip`.
|
|
|
|
From the admin UI: **Admin → Backup & Restore** (`/admin/backup/`) — runs in the
|
|
background with a progress bar, shows what's missing or orphaned, and lets you
|
|
download, verify, restore or delete archives.
|
|
|
|
From the command line:
|
|
|
|
```bash
|
|
flask backup create --label pre-docker
|
|
flask backup list
|
|
flask backup verify lim-backup-....zip
|
|
flask backup restore lim-backup-....zip --mode fresh
|
|
```
|
|
|
|
Stored paths are normalised to forward slashes, so a Windows-made backup
|
|
restores cleanly onto Linux — which is how the move to Docker works.
|
|
Full runbook: [`docs/BACKUP_AND_MIGRATION.md`](docs/BACKUP_AND_MIGRATION.md).
|
|
|
|
## 🐛 Troubleshooting
|
|
|
|
### Database Errors
|
|
```bash
|
|
# Reset database
|
|
rm lego_instructions.db
|
|
flask --app run.py init-db
|
|
```
|
|
|
|
### File Upload Issues
|
|
```bash
|
|
# Check permissions
|
|
chmod -R 755 app/static/uploads
|
|
```
|
|
|
|
### Brickset Connection Issues
|
|
- Verify API credentials in `.env`
|
|
- Check Brickset API status
|
|
- Review application logs
|
|
|
|
## 📝 License
|
|
|
|
This project is licensed under the MIT License.
|
|
|
|
## 🤝 Contributing
|
|
|
|
Contributions are welcome! Please feel free to submit a Pull Request.
|
|
|
|
## 📧 Support
|
|
|
|
For issues and questions:
|
|
- Open an issue on Gitea
|
|
- Check existing documentation
|
|
- Review Brickset API docs
|
|
|
|
## 🙏 Acknowledgments
|
|
|
|
- [Brickset](https://brickset.com) for the excellent LEGO database API
|
|
- [Bootstrap](https://getbootstrap.com) for the UI framework
|
|
- [Flask](https://flask.palletsprojects.com) for the web framework
|
|
|
|
---
|
|
|
|
**Built with ❤️ for LEGO enthusiasts**
|
|
|
|
Created by: Jess Rogerson (yelling commands at Claude.AI)
|