Smart media tagging and face recognition with on-premises machine learning models.
This app goes through your media collection and adds fitting tags, automatically categorizing your photos and music.
- 📷 👪 Recognizes faces from contact photos
- 📷 🏔 Recognizes animals, landscapes, food, vehicles, buildings and other objects
- 📷 🗼 Recognizes landmarks and monuments
- 👂 🎵 Recognizes music genres
- 🎥 🤸 Recognizes human actions on video
⚡ Tagging works via Nextcloud's Collaborative Tags
- 👂 listen to your tagged music with the audioplayer app
- 📷 view your tagged photos and videos with the photos app
Model sizes:
- Object recognition: 1GB
- Landmark recognition: 300MB
- Video action recognition: 50MB
- Music genre recognition: 50MB
Ethical AI Rating
Rating for Photo object detection: 🟢
Positive:
- the software for training and inference of this model is open source
- the trained model is freely available, and thus can be run on-premises
- the training data is freely available, making it possible to check or correct for bias or optimise the performance and CO2 usage.
Rating for Photo face recognition: 🟢
Positive:
- the software for training and inference of this model is open source
- the trained model is freely available, and thus can be run on-premises
- the training data is freely available, making it possible to check or correct for bias or optimise the performance and CO2 usage.
Rating for Video action recognition: 🟢
Positive:
- the software for training and inferencing of this model is open source
- the trained model is freely available, and thus can be ran on-premises
- the training data is freely available, making it possible to check or correct for bias or optimise the performance and CO2 usage.
Ethical AI Rating
Rating Music genre recognition: 🟡
Positive:
- the software for training and inference of this model is open source
- the trained model is freely available, and thus can be run on-premises
Negative:
- the training data is not freely available, limiting the ability of external parties to check and correct for bias or optimise the model’s performance and CO2 usage.
Learn more about the Nextcloud Ethical AI Rating in our blog.
After installation, you can enable tagging in the admin settings.
Requirements:
- php 7.4 and above
- App "collaborative tags" enabled
- For native speed:
- Processor: x86 64-bit (with support for AVX instructions)
- System with glibc (usually the norm on Linux; FreeBSD, Alpine linux and thus also the official Nextcloud Docker container and Nextcloud AIO are not such systems)
- For sub-native speed (using WASM mode)
- Processor: x86 64-bit, arm64, armv7l (no AVX needed)
- System with glibc or musl (incl. Alpine linux and thus also the official Nextcloud Docker container and also Nextcloud AIO)
- ~4GB of free RAM (if you're cutting it close, make sure you have some swap available)
- This app is currently incompatible with the Suspicious Login app due to a dependency conflict (ie. you can only have one of the two installed)
The app does not send any sensitive data to cloud providers or similar services. All processing is done on your Nextcloud machine, using Tensorflow.js running in Node.js.