Recognize

Highest Nextcloud version

Nextcloud 31
Show all releases

Community rating
Author

Marcel Klehr

Last updated

1 week, 2 days ago

Categories

Multimedia, Organization, Search

Resources
Interact
Report problem Request feature Ask questions or discuss

Smart media tagging and face recognition with on-premises machine learning models. This app goes through your media collection and adds fitting tags, automatically categorizing your photos and music.

  • 📷 👪 Recognizes faces from contact photos
  • 📷 🏔 Recognizes animals, landscapes, food, vehicles, buildings and other objects
  • 📷 🗼 Recognizes landmarks and monuments
  • 👂 🎵 Recognizes music genres
  • 🎥 🤸 Recognizes human actions on video

⚡ Tagging works via Nextcloud's Collaborative Tags

  • 👂 listen to your tagged music with the audioplayer app
  • 📷 view your tagged photos and videos with the photos app

Model sizes:

  • Object recognition: 1GB
  • Landmark recognition: 300MB
  • Video action recognition: 50MB
  • Music genre recognition: 50MB

Ethical AI Rating

Rating for Photo object detection: 🟢

Positive:

  • the software for training and inference of this model is open source
  • the trained model is freely available, and thus can be run on-premises
  • the training data is freely available, making it possible to check or correct for bias or optimise the performance and CO2 usage.

Rating for Photo face recognition: 🟢

Positive:

  • the software for training and inference of this model is open source
  • the trained model is freely available, and thus can be run on-premises
  • the training data is freely available, making it possible to check or correct for bias or optimise the performance and CO2 usage.

Rating for Video action recognition: 🟢

Positive:

  • the software for training and inferencing of this model is open source
  • the trained model is freely available, and thus can be ran on-premises
  • the training data is freely available, making it possible to check or correct for bias or optimise the performance and CO2 usage.

Ethical AI Rating

Rating Music genre recognition: 🟡

Positive:

  • the software for training and inference of this model is open source
  • the trained model is freely available, and thus can be run on-premises

Negative:

  • the training data is not freely available, limiting the ability of external parties to check and correct for bias or optimise the model’s performance and CO2 usage.

Learn more about the Nextcloud Ethical AI Rating in our blog.

After installation, you can enable tagging in the admin settings.

Requirements:

  • php 7.4 and above
  • App "collaborative tags" enabled
  • For native speed:
    • Processor: x86 64-bit (with support for AVX instructions)
    • System with glibc (usually the norm on Linux; FreeBSD, Alpine linux and thus also the official Nextcloud Docker container and Nextcloud AIO are not such systems)
  • For sub-native speed (using WASM mode)
    • Processor: x86 64-bit, arm64, armv7l (no AVX needed)
    • System with glibc or musl (incl. Alpine linux and thus also the official Nextcloud Docker container and also Nextcloud AIO)
  • ~4GB of free RAM (if you're cutting it close, make sure you have some swap available)
  • This app is currently incompatible with the Suspicious Login app due to a dependency conflict (ie. you can only have one of the two installed)

The app does not send any sensitive data to cloud providers or similar services. All processing is done on your Nextcloud machine, using Tensorflow.js running in Node.js.

Releases

Nextcloud version Stable channel Nightly channel All releases
31 9.0.0 - 31
30 8.2.0 - 30
29 7.1.0 - 29
28 6.1.1 - 28
27 5.0.3 - 27
26 4.3.2 - 26
25 3.8.0 - 25
24 2.2.2 3.0.0-beta.3 (Unstable) 24
23 2.1.2 - 23
22 2.1.2 - 22
21 1.12.0 - 21
20 1.12.0 - 20

Comments

Mace Moneta
Good
Ok
Bad
6 months, 4 weeks ago

Every update seems to require that the models be re-downloaded, but updating the app doesn't initiate that; it requires a manual 'occ recognize:download-models'. That seems excessive.

Johannes Rohr
Good
Ok
Bad
8 months, 1 week ago

While I was excited about this feature coming to Nextcloud, I found the result underwhelming. Automated photo tagging generated mostly useless output, with tags being only remotely related to the photo's contents, as any human viewer would instantly recognize. For me, at this stage, it is not even a proof of concept, but it feels more like pre-alpha software. At the same time, it explodes Nextcloud's "activities" sql table to several gigabytes, putting quite a bit of load on the server. At the current stage, this is approximately as useless as Tesla's "full self driving". I guess, this is mostly a response to the AI craze everywhere, but as so many other AIs, it is, at least at this stage, it is, in my experience, mostly a failure.

David Swanson
Good
Ok
Bad
10 months, 1 week ago

I have NextCloud (AIO) All-In-One installed on a Raspberry Pi 4 8GB that my families uses. For family use, it's perfect. It's all we need. I didn't expect much from this Recognize Extension. I thought it wouldn't work, and I would uninstall it as quickly as I installed it. But, it was really to install, and it actually works on the Pi 4. The only thing extra I had to do was install NodeJS on the Pi. That took all of a minute or so. Then added the directory in the extension settings /usr/bin/node And it works. I love it! Thank you to the developer!

Martin Zbořil
Good
Ok
Bad
1 year, 6 months ago

I am fascinated with the Recognize, it has a high accuracy but also makes very interesting false positives, for example making a 'face' group from people in hoods or grouping all 'animated' faces together.. I was trying to find out which face Recognize means, so i splitted a face into faces with one image, it turn out it was the same face, but Recognize found few more and decided to make a single face for each of the images with that person, -interesting- wtf.

issues are that many pictures under one 'face' group leads to the face stopping from loading, another, smaller issue is lack of information about the face Recognize found in the image (it used to be there in version 2.*)

... its not yet perfect, but getting there.

György Kiss
Good
Ok
Bad
1 year, 7 months ago

This app is incredible! It works really well. Very easy to set up. It takes a long time for the first recognition, but man, is it worth it! I expected face recognition working well, but I did not expect the tags feature working very well also! It created tags I did not think is possible. Super useful!

Dakota Gravitt
Good
Ok
Bad
1 year, 7 months ago

Works fairly well, with decent performance and plenty of options to limit performance impact also.

Anonymous
Good
Ok
Bad
1 year, 7 months ago

Development (bots not counted in): active

Issues: more solved than unresolved ones

Project management: canban

Documentation: wiki

Localization: continuous

Releases: frequent (changelog)

ABC Taylor
Good
Ok
Bad
1 year, 8 months ago

Too buggy to be useful. Does not appear to work with NC 26 unfortunately.

Gabriel Androczky
Good
Ok
Bad
1 year, 9 months ago

Simple to install and use (had to force request it to download the pre-trained models though when I first installed it), does its job nicely in the background. This app works flawlessly in my environment and is simply doing its job. I can't compare to other solutions built for Nextcloud, but it was important for me that none of my pictures would travel out to be classified and analyzed and this works nicely with this solution. If Nextcloud is configured correctly, it will do its job just fine. Running 4.0.1 since today, completely satisfied.

Hendrik
Good
Ok
Bad
1 year, 10 months ago

Very easy setup. After that, the app causes a lot of problems for me.

The recognition is very inaccurate. Audiobooks (which are also stored on my nextcloud) always gets detected as several music genres (from metal to techno). The face recognition is highly inaccurate compared to the other face recognition app available in the Appstore.

The object detection ist very "okayish", also does not produce very accurate results. After a few days of use, the app makes my server crash regularily, tried to debug, which is hardly possible.

So all in all, removed it after a few days and switched back to the "face recognition" app. May give it another try in the future

Reuben Shea
Good
Ok
Bad
2 years ago

The app itself? Works as advertised, added tag options for my music and photos and I can search for things based on tags as well so it becomes a very usable feature in theory. Processing can be fairly slow (albeit I had to use WASM mode); as an example the recommended for WASM mode facial recognition is 10 images per minute (10x+ speed for regular mode depending on your machine). Not unreasonable, honestly, but there is an idea for those who need it.

That said, the actual models used for recognition are... rough. I mostly use mine for music and object recognition. Object recognition is okay, a bit scattered but generally fine. Music was maybe 60/70% accurate with my small 100 song test library (had 121 songs and 97 got labels). Maybe I just have weird music (which admittedly I do) but I am not sure how useful that is.

I'll keep using it installed and hope it improves; it is still relatively new. I would love to see the models get a bit more robust.

Rachel Wood
Good
Ok
Bad
2 years, 1 month ago

A very idea and it works but in my opinion has an issue that makes it unusable and hence I will probably remove the app. This is not the place to report bugs or request features so I will leave my comment as "lacking necessary features"